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0. Introduction 

This report draws together the diverse strands involved in developing a unique computer-based system to 
stage and manage Hodgkins Disease (HD). Those of us who worked on the final version of this project 
included two hematologists, a computer scientist, and a statistician. We have all contributed to this report, 
and our respective sections reflect not only our own special interest in and approach to the problem, but also 
the type of readers with whom we hope to communicate. We expect mat many will not read this report "in 
toto", but that others working on similar problems may find particular sections helpful and interesting. In 
addition, because the program is not being maintained, we have included much data to insure that 
information from our extensive patient database is not tost 

The report begins with a general discussion of the principles of management of HD, covered in sufficient 
detail to explain the problem to mose not familiar with this field: This section is aimed towards the 
non-physician or the physician who is not involved with HD management; it explains the rationale for a 
structured approach to the problem. 

Section 2 describes the patient database, and how this is used to predict the likelihood of the various stages of 
HD. A general discussion of Bayes Rule then leads to a description of the particular way this rule is applied 
to revise the probabilities of pathologic stage prior to management decisions. 

Section 3 deals with the technique of decision analysis. No previous knowledge of decision analysis is 
assumed, and detailed descriptions of all the steps involved are given, with the examples based on the HD 
system. The latter part of this section requires an understanding of Bayes Rule as described in section 1 

Sections 4 and 5 are concerned with the statistical validity of the program. After describing, the assumptions 
made in developing the program, this section deals with the way both these assumptions and the general 
validity of the program were tested. Details of the methods used are supplied, and wider application of the 
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method is discussed. 

Section 6 is included to show exactly how the Bayesian and decision analysis techniques are applied. It gives a 
detailed description of a typical patient session, including all the special features which can be used to tailor 
the decision plans to a particular patient and the experience of a particular hospital 

In contrast, sections 7 and 8 describe more general applications of our system. We use the program to 
approach some important management dilemmas in HD and we draw conclusions pertinent to large number 
of patients. Although these sections contain information mainly of interest to the practising hematologist, the 
discussion of decision-making thresholds has a more general application. 

Rigorous application of Bayes Rule and decision analysis to the inexact science of clinical management is 
bound to result in some interesting problems. These are addressed in section 9, in which we discuss the 
limitations we found in this approach. 

Section 10 is written as a guide to computer programmers interested in the implementation of such a program. 
The program is described in sufficient detail to enable a similar program to, be set up. Certain original and 
sophisticated techniques developed for this program are described. 

Finally, there are four appendices. The first is a comprehensive list of prior probabilities for the basic clinical 
findings, the conditional probabilties which are used to modify these prior probabilities, and posterior 
probabilities for every combination of clinical findings, all of which are derived from our 1200 patient 
database. We feel that these will be very valuable to the physician managing HD. The second appendix gives 
the general format of the statistical test used to evaluate probabilities of stage as described in section 5. The 
short Appendix 3 documents the information used in the program to derive the conditional probabilities of 
stage from the lymphangiogram, and shows how these probabilities are derived from the data. Finally 
Appendix 4 gives an annotated trace of the decision tree analysis program in operation. 
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1. Hodgkins Disease - Modern Approaches to Staging and Treatment 

Although Hodgkins Disease (HD) is a malignant disease of the lymphoid system, a malignant lymphoma, its 
biologic course is different from me other lymphomas, and from neoplasms in general. In the majority of 
patients the disease arises in a single lymph node area, typically cervical, and spreads by lymphatic channels to 
involve contiguous lymph node areas in an orderly way. Local spread of disease to structures contiguous to 
affected lymph nodes is noted, particularly with involvement of mediastinal and hilar lymph nodes; 
occasionally the disease may begin in a localized extranodal site. Splenic involvement usually occurs as the 
disease progresses, and is thought to result from hematogenous spread, since the spleen has no afferent 
lymphatics. Hematogenous spread to other extranodal stnatures such as bone marrow, liver ox lung does not 
usually occur until late in the disease. 

The extent of the disease determines the type of treatment which is indicated. To achieve uniformity in 
describing this extent the Ann Arbor staging system was adopted in 1970, and is now well-established 
(Carbone et al 1971) (Table 1.1). This stage of the disease, together with the presence (B) or absence (A) of 
the specific symptoms is the most important guide to prognosis, with the outlook becoming less favorable 
with advancing stage, and if symptoms are present 

There is now good evidence that proper treatment of HD in certain presentations results in long term 
remissions and probably cure. For localized disease radiotherapy may be curative in a high proportion of 
patients if all the tumor is encompassed with a tumoricidal dose of radiation - usually in the vicinity of 
3500-4000 rad; for disease which has spread beyond the lymph nodes and spleen, combination chemotherapy 
has given a high proportion of patients such prolonged remissions mat they may.be considered cured. For 
other stages, where neither radiotherapy nor chemotherapy when given alone has proven extremely 
successful, combinations of these two modalities are being tried to effect a cure. 

The diagnosis of HD often marks the beginning of a scries of increasingly invasive diagnostic tests to 
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determine this stage in order to select treatment This intensive staging may include an intravenous 
pyelogram, percutaneous bone marrow and liver biopsies, a gallium scan, a lymphangiogram, and 
computerized tomography; since all these teste have an appreciable raise negative rate, and, with the 
exception of the biopsies, all have a false positive rate, (Table 12) the series often culminates m an exploratory 
laparotomy, with extensive lymph node sampling and splenectomy. If the extent of disease is established by 
laparotomy, or by histological confirmation of disease in extranodal structures by a biopsy, the patient is said 
to be "pathologically-staged". Staging without exploratory laparotomy, where biopsies of extranodal sites are 
negative for disease, is termed "clinical staging". 

Our bask purpose in applying the techniques of decision analysis to HD has been to tailor staging 
investigations to the individual patient, and thus to use die minimum number of investigations to select 
treatment This has involved studying a large series of patients in order to draw general conclusions about the 
management of HD which could tisen be accessible to a wide audience of physicians, with or without access to 
computer facilities for individual patient study. 

For any patient the value m performing an investigation is related to mree factors: 

1. the likelihood that the patient has die abnormality for which (s)he is being tested. 

2. the sensitivity or true-positive rate of the test 

3. trie spedffcky or true-negative rate of the test 

All too often in HD the whole battery of available tests are used without analyzing then- value in a particular 
patient and without considering that tiiey may provide overlapping information. There are a number of 
compelling reasons for performing the minimum number of teste required: 
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Table 1.1. The Ann Arbor Staging System for Hodgkin's Disease 



STAGE 
Stage I 



Stage II 



Stage III 



Stage IV 



CHARACTERISTICS 
Involvement of a single lymph node region, or of a 
single extralymphatic organ or site (designated by E). 

Involvement limited to one side of the diaphragm either 
of two or more lymph node regions, or loeaBzed^nVdfveinent 
of an extralymphatic (E) site and one or more lymph node regions. 

Involvement of lymph node regions on both sides of the 
diaphragm, which may include I6caai^tow*vem1intt>fan 
extralymphatic organ or spleen. 

Diffuse or disseminated involvement of one or more 
extralymphatic organ, or any liver involvement, with or 
without associated lymph node involvement 



Each case is further classified as: 



B 



if asymptomatic 

if any of the following are present: 

- unexplained weight loss of more than 10% 
ofbody weight in the preceding 6 months. 

- unexplained fever, with temperatures above 38 C. 

- night sweats. 



modified from Carbone et al 1971. 



Table 1.1 False Positive and False Negative Rates of Tests 



FP(%) FN(%) REF 



Gallium scan (abdnodes) 

Lymphangiogram 

Liver scan (abnormal uptake pattern) 

Spleen scan (i) filling defects 

(ii) size >15cm 
Computerized tomography (abd. nodes) 



5 


49 


McCaffrey(1976i 


13 


16 


Appendix III 


23 


38 


Mildner(1973) 
Ljptpn(1972) 





81 


Silverman(I972) 
Mildner(l973) 


7 


60 




17 


13 


Lec(1977), Best(1977) 

Alcorn(1977) 

Breiman(1978) 
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L with the exception of the gallium scan and computed tomography. aB the tests have a mortality 
risk. This may be extremely remote ia me case of a bene marrow biopsy, but mere is a small 
mortality risk for such tests as the lymphangkgiam and laparosa>py, and an appreciabk risk for the 
staging laparotomy. Data collated from me medical literature for 2345 patients who underwent 
staging laparotomy is summarized in Table 1.3, and shows a mortality rate of 0.5% overall lor the 
laparotomy and its immediate post-operative period. 

2. all tests, again with the above exceptions, have a defiake risk of morbid con^lkadons; here too 
die highest incidence of serious complications is seen after stagmg laparotomy, where 6.7% of 
patients experienced major complications as serious as pulmonary embolism, subphrenic abscess, 
intestinal obstruction etc, and a further 15% had less severe ooe^pticatiaas. This data is also 
summarized in Table 13. 

3. All tests are expensive. A ftiH work-up which uses aft die tests could cost in the vicinity of 
$10,000. Individual costs lor die different procedures are given in Table 3 J. 

Information on the specificity and sensitivity of the tests used to stage HD is readily available in the medical 
literature. Information on die likelihood of positive findings for me various tests under different conditions, 
however, is not 

In the following section we describe how we derived mis information from data obtained Horn 
laparotomy-staged patients. We also describe how, from me same data, we use Bayes* Rule to predict the 
probabilities of die different stages for a particular patient with Hcdgkins disease, so that decisions about 
stagmg investigations and treatment may be made using die teclmk]ue tf deriskm anahws. 
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Table 1 J. Mortality and Morbidity from Staging Laparotomy for Hodgkins Disease 



Reference 


No. of 


No. of 


No. of 




Patients 


Deaths 


Major 
Complications 


Aisenberg et al (1974) 


100 





2(2%) 


Andrassyetal(1977) 


76 





1(1%) 


Beretta etal (1976) 


110 





1(1%) 


Brogadiretal(1978) 


90 





16(18%) 


Bruntsch etal (1977) 


275 





28(10%) 


Cannon etal (1975) 


400 


1 


15(3.7%) 


Coleman etal (1976) 


31 


1 


7(23%) 


Ferguson etal (1973) 


31 


1 


1(1%) 


Gamble etal (1975) 


139 


1 


5(4%) 


Garcia etal (1971) 


20 





0(0%) 


Gazet(1973) 


65 


1 


4(6%) 


Hermreck etal (1975) 


50 





13(26%) 


Jelliffe etal (1970) 


22 





1(5%) 


Lowenbraun et al (1970) 


12 





0(0%) 


Marston (1972) 


60 


2 


4f7%) 


Meeker etal (1972) 


30 


2 


4(13%) 


Mitchell etal (1972) 


45 


1 


3(7%) 


Paglia etal (1973) 


51 





6(12%) 


Piro etal (1973) 


114 


1 


6 (5%) 


Poulsen etal (1977) 


91 





5(5%) 


Prosnite etal (1972) 


40 





4(10%) 


Roberts etal (1976) 


82 





3(4%) 


Rozman etal (1973) 


56 


1 


2(4%) 


Smith etal (1972) 


70 





2(3%) 


Sutcliffe etal (1976) 


98 





19 (1%) 


Urlaub etal (1979) 


107 





5(4%) 


Zarembok etal (1972) 


30 





0(0%) 



Total 



2345 



11(0.5%) 



157(6.7%) 
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2.1 

A great deal of information about die patterns of spread ofHodgkius disease (HD) has been amassed in (he 
decade since exploratory laparotomy has been regulariy used to stage &e disease; from this information has 
come our knowledge of me predictable pattern of spread of HD according to normal lymphatic distribution. 
The different histological subtypes have been found to have different patterns of spread, with die degree of 
dissemination increasing from lymphocyte predominant (LP) to nodular sclerosis (NS) to mixed cellularity 
(MQ to lymphocyte depleted(LD). We know also that the presence of symptoms is related to me spread of 
disease, with symptomatic (B) patients having more advanced disease man asymptomatic (A). Other factors 
known to affect me extent of the disease are the sex and the age of die patient The younger age group, 
corresponding to die first incidence peak in die characteristic biapdal age:mcidence curve of HO, shows an 
almost equal malerfemak ratio, a predominance of the NS histologic pattern, a higher frequency of 
mediastinal involvement and a more benign clinical course. In contrast, die older group, corresponding to die 
second peak of die age:incidence curve, shows a higher matefcmak ratio, and a number of inter-related 
clinical features including a greater proportion of die MC hatpiogical fc/pe, a much higher incidence of 
mfradiaphragmatic involvement at presentation and a more aggressive elm wd course (Gutensohn et al 1977). 
The pattern of involved supradiaphragmatic lymph nodes is related to (he frequency of associated 
mfradiaphragmatic disease; left cervical node involvement is asso c iate d with a higher incidence of disease 
below the diaphragm whereas with mediastinal involvement a lower incidence is seen. 

We have used data from approximately 1200 patients who have been pao^ologk^ staged, to help predict die 

■ ■ ■ V 

UkcKwod of me various stages of Hodgttos disease according to die basic cfimcal data known about me 



All information about the following findings which are available for a given patient have been collected and 

stored: 

Histologic subtype 

Presence (B) or absence (A) of symptoms 

Sex 

Age 

Specific supradiaphragmatic lymph node groups Involved 

Alkaline phosphatase levels 

Clinical splenomegaly 

Absolute and percentage lymphocyte count 

Liver/spleen scan 

Percutaneous bone marrow biopsy 

Percutaneous liver biopsy 

Gallium scan 

Computer-assisted tomography (CAT scan) 

Lymph angiogram 

Peritoneoscopy 

Laparotomy 
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A typical string of patient data may be seen below: 



{(SOURCE TUFTS) (ID XX) (AGE 20) (SEX FEMALE) (HISTOLOGY IS) (A-OR-B A) 
( LOCATION-PRESENT IMS-NODES (LEFT-NECK RIGHT-RECK)) 
(MEDIASTINUM NEGATIVE) (LIVER- SCAR NEGATIVE) 
(SPLEEN-SCAN NEGATIVE) (SPLENOMEGALY NORMAL) 

(ALK-PHOS (BOOANSXY 3.0)) (GAL POSITIVE-ABO-NODES) (LAG POSITIVE) 
(BH8X NEGATIVE) (LBX UNKNOWN) 
(LAP ((LAP-SPLEEN NEGATIVE) 

(LAP -MARROW NEGATIVE) 

(LAP-LIVER NEGATIVE) 

(LAP-NODES POSITIVE)))) 



Patient data from the following institutions was obtained: 

Tufts-New England Medical Center 91 patients 

Joint Cento- for Radiation Therapy 89 patients 

Massachusetts General Hospital inpatients 

Stanford Division of Radiation Therapy 504 patients 

Harvard School of Public Health 31B] 



The remainder came from individual patient data in the medical literature by Bearman etal (1978), Hanks et 
al (1971), JeUiffe et al (1970). Lowenbraun et al (1970X Mitchell et al (1972), Prosnitz et aHia72LZarembok 
etal(1972). 

The patient with me uncommon mfradiaphragmatic presentation of Hodgktns disease was poorly represented 
in the initial database. Furthermore, in mis type of patient certain test results have a different meaning in 
terms of staging, eg. such findings as involved abdominal lymph nodes or involved spleen define die patient 
as having stage II disease only. For these two reasons, we have excluded the patients with an 
mfradiaphragmatic presentation from me database, and do iiot attempt to analyze dus type of case. 

Initially die proportion of patients in each pathological stage was calculated from die raw data. Stages I and II 
were analyzed together, as therapeutic decisions for these stages are almost always identical. Each finding was 
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then assessed to see whether it was helpful in predicting the psmological stage. Certain findings - alkaline 
phosphatase and clinical assessment of splenomegaly - were not found to be helpful in this respect, so they 
were not used in the final version of the program. 

For those findings which were found to be helpful in distinguishing among the different stages, the number of 
patients in each pathological stage was ascertained and was expressed as a proportion of 1.0 e.g. for the 
finding Histological Subtype: 



Histological 
subtype 


Pathological 
stage 


No of 
patients 


Proportion 
of patients 


LD 


I+II 

III 

IV 


7 
9 

e 


0.32 
0.41 
0.27 


LP 


I+II 

III 

IV 


49 

14 

2 


0.75 
0.22 
0.03 


MC 


I+II 

III 

IV 


106 

112 

36 


0.42 
6.44 
0.14 


NS 


I+II 

III 

IV 


461 

236 

44 


0.62 
0.32 
0.06 



One can use this data to calculate the probability of being in a particular pathological stage, given the 
histological subtype, e.g. me probability that a patient demonstrating the NS subtype will be m stage III is 
0.32, or simply stated P(Hl|NS) = 0.32 

Similarly the conditional probabilities of STAGE given SYMPTOMS can be obtained from the database: 
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Pathological 
stag* 


«o of 
pattest* 


ProtafciHtv 


MI 
III 

IV 


486 

MS 
31 


6.6S 

6.3 

6.64 


I+II 

III 

IV 


161 
167 

63 


6.41 
6.4t 

0.17 



From this one may derive the piobabaity of STAGE fiwn SYWTOl^ e$ FflVjA) = 0jQ4. Data can be 
similarly expressed for all the {hidings entered. 

If the findings were all independent of one aao&er, then one could find the probability of having a 
combination of findings in «age (I +11). stage HI, or stage rv for any patienrshnply by multiplying togefter 
me conditional probabilities of each finding given the stage. For example, for an asyniptomatic (A) male 
patient who has die MC histological subtype: 

p(A HALE HC|III) would be p(A| III) X p(KALEjIII) X pJMC|III) 

Clinical information, howerer, suggested dot some findings were nicely to be interdependent Chi-squared 
testing (see Chapter 4) did confinn mis, and dependence was shown among die 6^ pause's sex, u^ presence 
or absence of symptoms and die histological subtype, tho*ua>* was not seen 4B»ong the other findings used 
by the program This dependence precluded the simple type of calculation shown above Instead, poor 
probabilities of stage were calculated from the data for die triad of interdependent findings ~ die patient's 
sex, the presence or absence of symptoms and die histological subtype - making sixteen different 
combinations of prior findings. For any patient, Bayes' rue was then used to modify these prior probabilities 
by die conditional probabilities of FINDINGJSTAGE for me subsequent findings used in me program - the 
patient's age, involvemcnt/noa-involvement of cervical and mediastinal lymph nodes, and die results of die 
various tests, including liver-spleen scan, bone marrow and liver biopsies, gallium scan, and lymphangiogram. 
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2.2 The Use of Bayes Rule To Revise Probabilities of Stage in Hodgkins Disease 

Bayes Theorem is a mathematical method for modifying probabilities when new information is available. We 
have used this method to revise the probabilities of stage, as we take into account the basic clinical 
information known about the patient and the results of testing. See section 2.2.2. 

2.2.1 General Application of Bayes Role 

Consider a test with two possible outcomes (T+ or T-) which is used to modify the probability of a patient 
being in a disease state, D. If P(D) is the probability of the disease state prior to the test, the revised 
probability P(D|T+) Le. the probability of D given a positive outcome, and of P(P|T-), i.e. the probability of 
D given a negative outcome, can be expressed as follows: 

p(d|t+) - P(b) piu'ip ! iT p(3!) p(H-*>) 
p(dit-) ■ P(b) P(H P ftS° y + Pf Vffl KH-fl) 

Where P(T+|D) = Probability of a positive test, given the disease 
P(~D) = probability of no disease 

P(T+|~D) = probability of a positive test, given no disease, i.e. 
the false positive rate. 

P(T-|~D) = probability of a negative test given no disease 

P(T-|D) = probability of a negative test given rfiseaae, I.e the 
false negative rate. 

Knowing the prior probability of a disease state, P(D), and the false positive and false negative rates of a test 
used to detect it, one can use Bayes' rule to revise P(D) according to the outcome of the test 



December 1981 



. /j . Hodgkins Decision Analysis 



HI Use efBayes Rate ■ the 

boar program we have naedlafesErfe to iew*^ 

account bc4h bask; dinicalmfaniati^ 1^ probabildes 

for each stage are those derived directly from the database fix (be dure banc findings which ha ve been found 
to be interdependent, namely histologic subtype, sex and the presence or absence of symptoms. Prior 
probabilities for each of the sateen possibk combinations of the fmdmgs are grrom Appendix I. 

To demonstrate how we apply Bayes Rate we will show die calculations involved in the case demonstrated in 
Chapter^. This patient is a male with B symptoms - weight loss - and me nodular sclerosis histologic 
subtype. The prkvprobabm^ of stage for ^padent are: 



Stages I-t-II 


0.34 


Stags III 


0.69 


Stag* IV 


Q.lt 



We wish to revise these probabilities of stage knowing the patient is in lb* age group 12-39 years. For (bis 
we need to know the conditional probabilities of AGEJSTAGE which have been calculated from me 
database, and which have been shown to be independent of other findings. These conditional probabflities 
are; 

Younger than 12 years l+U 0.03 III P~££ IV 0.03 
12 to 39 years I+H 9.84 HI 0.74 IV 0.40 

Older than 39 years I+II 0.14 III 0.23 IV 0.47 

To update the prior probability of stage (I + H) for example 

p(I+II|AGE12-39) - 

p(I+II).p(AGE12-39|(I+II) 
p(I+II).p(A&E12-39t(I+H) + p(III)p(A6C12-39t(HI) + p{IV) .p(A6£12-391(IV) 

The term p(I + H).p(agcKl + H» corresponds to me "true positive" if one considers age in die sense of a 
Dectmbermi -M- Bod&ka Dtcim* dm** 



"test", with being in the 12-39 age group a "positive" result afldtae other age groups a "negative" result This 
makes the other terms in the eqation - p(III).p(AGm2-39Rffl>&ta p(IVKp^GEl2-39|(IV)-1he equivalent of 
false negative results. Substituting the probability results in the equation: 

p(I+II)|AGE12-39 » 0,34X0.84 

(0.34 X 0.84) + (0.6 X 0.74) + ((0.16 X 0.46) 
0.3926 

Similar calculations are carried out to revise the prior probabilities of stage III and stage IV according to the 
conditional probabilities of agelstage. 

These posterior probabilities now become the new "prior" probabilities which can then be revised by the next 
piece of information available, suchas the specific sitesof supm-diapluip^ticlpaph node involvement. 

Bayes Rule is applied to the tests used in staging HD according to the general formula given in section 12.1. 

For example, suppose we have calculated that the probability of abdominal node involvement in a patient 
with supradiaphragmatic HD is 0.30. A gallium scan is found to be positive for abdominal nodes, and we 
wish to know how this positive result has modified our initial probability. 

P(+NODES) = 0.30 therefore p(-HODES) = 0.70. 

From the literature we find that when the gallium scan is used to detect involved abdominal nodes in HD: 

False positive rate =■ p(+te$t|-N00BS) « 0,18 

False negative rate = p(-test|+NODES) =0.50 

where the symbols + and - are used for* positive and negative, i.e. 
+N0DES are lymph nodes involved with HD. 

From these we deduce: 
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Ttwm positive rntn * p(+t*st|+80efcS) » fr.68 
Twin n«f**.t«« tntn » p(-t«*tiHKBES> -8.6ft 
We cm now cafcubte the revised probabffity 



p(+ft0BES) p(+tMt|-HK»ES) 
p(+«OOES(+te$t) « •-!--*- 

p(+MOOCS).p(+test|+»00£S) + p(-BOO€S).p(+te$tI-iOOES) 

.3 X .6 
(.3 X .5) + (.7 X .18) 

■ 0.68 or 68% 

The me of Bayes Rule here has revised the probability of abdominal node mKtfveffient upwards, from &30 to 
0.68 in the ligjit of the positive resuk tor gaffium scan. 

In om* application of Bayesiaa me&ods to fte staging of ftbdgkms dfeease, we use the tests to modify 
information about a stage of fee disease, rather than the specific sites of invotvement, as demonstrated in die 
example above. If we wish to find the revised probability of stage HI discare 
gaffiumsc^isposith^wecanwiiretheb^iccalcidationac 



p(+€AL|III).p(III) 

p(III|+GAL) » —■ -■ — -— --i*.-^*-;— -•— :— — 

p(+6AL|I+II).p(I+II) + p(+€AL[III).p(III) + p(+6AL|IV).p(IV) 



However, me fiterature does sot contain data in the form of conditional probabilities of test results in certain 
stages of die disease. IMs-eafeufetioftv therefore nm6t be carrJedouttttwestages: 



p(+€AJHUIJ = p(+6AL{+«0B«S> p( -HOSES till) 

TRUE POSITIVE 
♦■ ■ 
p(+GAL|- MOOES) p(-IK»0£S|III) 
FAtSE POSITIVE 



The terms for p(+GAL|(+Il (or IV)) are expressed in the same way. The probability of positive and 



flmiwnH Iwf -JS> #ftvfe*to ftiM> Awgafc 



negative nodes in stage III is obtained from the database, Le. 



In stage IIIA: 

p(+ABDOMINAL-NODE-INVOLVEMENT) = 0.62 
p( -ABDOMINAL-NODE-INVOLVEMENT) = 0.38 



These lead to the conditional probabilities: 

p(+GAL|IlI) * 0.5 X 0.82 + 0.10 X 0.38 

TRUE POSITIVES FALSE POSITIVES 

= 0.31 + 0.038 

" 0.36 

That is the probability of finding a positive Gallium scan in a stage IIIA patient is 0.3S. 

Similar calculations are performed for all stage/symptom combinations for this test and all the other tests 
used, so that conditional probabilities of the test given stage and symptoms are available for the Bayesian 
calculations. For the gallium scan these are: 

NEGATIVE GALLIUM SCAN 

I+II A 0.90 B 0.90 

III A 0.65 B 0.66 

IV A 0.81 B 0.54 

POSITIVE GALLIUM SCAN 

I+II A 0.10 B 0.10 

III A 0.35 B 0.44 

IV A 0.39 B 0.46 
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23 

Conparing Ate incident of Bodgjdnsdi^^ 

curve with one peak in early aduh ire. arid a second SMiaiiiediiia^^ 

The two age groups defined by this carve show important differences, wife the age at presentation showing 

correlation with the extent of the disease. The younger group shows an approximately equal mate:female 

ratio, a predominance of the nodular sclerosis (NS) histologic patera and a more benign clinical course, with 

mecliastinal involvement quite common; in contrast, me older group shows a higher mate:female ratio, and a 

number of inter-related clinical features including a greater proportion of the MC histological type, a much 

higher incidence of mfradiaphragmatic involvement at presentation and a more aggressive clinical course. 

We have, therefore, used age as one variable for estimating die probabilities of stage in Hodgkin's disease 
patients. Conditional probabilities of AGE given STAGE are used in applying Bayes rule to calculate 
posterior probabilities of STAGE given AGE Use of this formula requires the assumption mat age is 
independent of toe findings used to calculate the prior probabilities. This assumption was checked for the 
arbitrarily-selected age groups - 0-15 years, 16-38 years, 3M5 years, aijd older tiw 45 y^^ - inWa^ used in 
the program by means of Chi square analysis, and found tenable (Eisen 1977). 

Subsequent analysis of the larger database showed mat there were essentially three age groups distinguished 
by distribution of stage: to 11 years, 12 to 39 years, and 40 years and over. Homogeneity was observed 
within each group, with some minor statistical fluctuation, and distinct differences were observed between 
groups (as determined by Chi square testing). 
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3. Decision Analysis 

Decision Analysis is a technique for evaluating complex decisions by breaking them down into their simplest 
component parts, quantitatively evaluating these, then reassembling them in a logical structure. 

Decision analysis involves four sequential steps: 

a) constructing the decision tree which displays, in chronological order, all the options for a 
particular decision, and every possible outcome of each option, whether determined by chance, or 
by further decision-making. 

b) evaluating the likelihood of those outcomes in the decision tree which are determined by chance. 

c) scoring each possible option, according to a system of utilities. 

d) evaluating all decision branches and selecting those with the highest utility. 

3.1 Constructing the Decision Tree 

The decision tree is displayed on a branching framework, and includes all the options among which one must 
choose, with all possible outcomes of each option displayed. Subsequent outcomes include both those 
determined by chance, and those which involve later decisions. 

At each branch point in the tree where a dedsiori must be matted asrnaflsqaare is used to denote a decision 
node. At any point in the tree where the outcome of a decision fe^fetermhiecVbytRance, asmafl Circle denotes 
a chance node. 

Below is shown the simplest type of decision tree used to Oris work, the basic TEST versus TREAT decision, 
in which a physician has to decide whether to perform a test to define the patient's disease more accurately or 
whether to treat (Rx) the patient without further ado. By convention, the decision tree is drawn with the early 
decisions on the left, the later outcomes on the right 
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TEST-1 



r-t 



TEST-X 



<S> 



Rx 



o 



^ 



Rx 



TEST-X 



NBQ 



Rx 



At decision node a, the decision is between treatment (Rx) or performing TEST-1. At chance node b, the 
patient may die (+) from the lest- this risk being the observed test n»rtaH^ At 

chance node c, having survived die test, the patient may have either a positive ora negative resuk for the test. 
At each decision node d, one must now, in trie light of mfonnauongained from the, test, make another TEST 
versus TREAT decision, for some other test 

The actual decision tree involved in the selection of test procedures and treatments in Hodgkins disease is 
very complex, because: 

(i) mere are a number of different tests among which one must choose - namely bone marrow 
biopsy (BMBX), percutaneous liver biopsy (LBX). peritoneoscopy with guided liver biopsy (PTX). 
gallium scanning for abdominal node involvement (GALL), lymphangiogram (LAG), and staging 
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laparotomy (LAP) - and one may choose any number of these. 

(ii) The order in which the tests are performed may be important A test sednehce stops with a 
positive bone marrow or liver biopsy which constitute definite evidence of disseminated (stage IV) 
disease, or with the staging LAP, which is considered for practical purposes as "perfect,; 
information", i.e. it is assigned neither a false positive nor a false negative rate and its results are 
always final. Following a negative bone mallow "or over biopsy, or after gallium scan or 
lymphangiogram further tests must be considered. 



Below we show part of the decision tree relating to the selection of tests, to demonstrate the recursive way in 
which any tests not yet performed are considered. This decision tree displays only one of the branches which 
occur at each decision node, following the initial decision to select bone marrow biopsy. Where further 
decision "branching" follows, the test name is followed by dots, e^. "GALL. . ." . A similar branching 
structure exists for each of the other initial tests, except for laparotomy, where results are final. 

LAP 

LAP r- 

LAP 



LAP 



LBX 



o 



^MBXI 



GALL 



LBX I 



LAP 
GALL 



GALL 



& 



LAG 



LAG 



LAP 



LAG 



LAG 



LAG 



the actual tree evaluated is even more complex since at every decision node, the decision involves not only 
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tamer TESTING, but mist abo include an 



of the TREATMENT option. The actual framework 



LAP 



LBX 



LAP 



LAP 



LAP 



o 



LBX 



W) 



GALL 



LAG 



GALL 



<r 






LAG 

——-•TIT 



LAP 



Rx 



WtLAG. 

Trx 



LAG 



Rx 



Rx 



The TREATMENT option may include any of a somber of deferent treatments; these include radiotherapy 
to all lymph-aode areas, knows as total nodal irradiation (TNI), radiotherapy which snares die pelvic lymph 
nodes, which is known as extended mantle irradiation (EM), and the combination chemotherapy regime 
MOPP. 

The TREATMENT branches for asymptomatic (A) and symptomatic (B) disease ait shown below. For A 
disease there are three treatments to choose amongst, for B patients only two m our ba^ dxision tree. The 
tree may easily be modified, however, to include other treatments. 



BeetmttrMKI 



■»- 
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A 
patient 



EM 



TNt 



MOPP 



B 
patient 



TNI 



MOPP 



It is also possible to compare single modality therapy, with regimens which include both irradiation and 
combination chemotherapy (combined modality therapy). If this option is selected, however, for proper 
comparison with the single modafity therapy, allowance must be made for "salvage" therapy (usually with 
MOPP) should relapse occur. Below we show the treatment branch when combined modality therapy is 
included in the treatment being evaluated for patients with symptomatic (B) disease. 
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<^ 



MOPP 



$ 



Ma 



TNI 



CoaiBtwd 

J MMHIOfl 



notapw »>MOPP 







i 



Hrf 



NO 



MOPP 



Continued 
I fWMMnn 

} Beiapee »> 



MOPP 



i 



No 



3J E?aliut«itiKUkda»edoftfceO*co»M 



la this program we dart wim those probabilities of die pathologic stage derived from the repealed use of 
Baycs Rule, according to the bask findings knows about tbe patient. Par example, fer a 20 year old 
symptomatic male patient, who has mixed cdhriarity (MQ Hodgkins diseaae involving the right cervical 
nodes and die mediastinum, and who has a aormal liver/spleen scan, the prior probab^ies are modified as 
shown in Table 3.1 before decision analysis is used to decide further i 



For this patient, these probabilities of stage derived from die bask clinical findings now become the "prior 
probabflities" for the decision analysis. As each tea b evahiated, Acsc pix)babilitjes are modified by: 



/WMMRCf ¥J9i 



Table 3.1. Results of Bayes Rule, Incorporating Basic Clinical Findings 

Finding Stage I+II Stage III Stage IV 

MMCB 0.20 0.44 0.36 

Age >11<40 years 

L cervical nodes -ve 

R cervical nodes +ve 

Mediastinal nodes +ve 

Spleen scan normal 

Liver scan normal 



0.2* 


0.60 


0i24 


0.30 


0.48 


0.24 


0.27 


0.42 


0.31 


0.30 


0.4* 


0.30 


0.43 


0.34 


0.23 


0.4ft 


0.31 


0.16 




-■ - . 





L the known mortality rate for the test (see decision node b in the first basic TEST versus TREAT 
decision tree in section 3.1). 

2. the known false negative and, where applicable, false positive rates for the tests, applied 
according to Bayes Rule. 

Suppose in the example, we are considering the branch of the tree which begins with bone marrow biopsy; 
note that this test has no false positive rate, and from the reported false negative rate, we have derived the 
following conditional probabilities: 

Probability of NEGATIVE bone narrow biopsy (-BHBX) with 

I+IIB ' 1.0, II IB * 1.0, IVB * 0.66 
Probability of POSITIVE bone marrow biopsy (+BM8X) with 

I+IIB = 0.0. IIIB « 0.0, IVB -0.44 
The basic probabilities are modified by Bayes rule, using these conditional probabilities, eg.: 
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p(IVl-BWX) 
p(I+II) p(-9*«X|I+II) + #(111) p(-B«M|III) + p(IV) p(HH*XfI¥} 



0.16 X 9.99 
0.48 X 1.0 + 9.37 X 1.9 + 0.19 X*9.99 



0.094 
- — ---t— * 0.99 
0.934 



0.094 
0.49 + 0.37 + 0.094 
Similar cafculations are carried out for stages 1+ H and III giving revised probabilities if the test is negative of 



I+II8 0.51 III8 0.40 IV9 0.99 

If positive, the test constitutes definite evidence of stage IV disease. These revised probabilities are entered 
mto the next stage of the decision tree. 



LAP 



LAP 



LAP 



LBX 



i J20: JS:.1 2 



.51:.40:.09M 



LBX 



, I BMBX JL 



GALL ofto.on.0 



LAG 



|.S£.41 



•.M&t* 



.GALL 



GALL 

Ottaon.0 



LAG 
Rx 



LAG 



Rx 



Rx 



LAP 



LAP 

j pif. n. i unf it ./ 



+ A LAG 
Rx 
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3.3 Assigning Utilities 

The utility is the value placed on a particular outcome. Utilities may be expressed on an arbitrary scale, or a 
measurable quantity may be used. We have used several different types of utilities to assess our decisions: 

i) Disease-Free Survival Five Years After Treatment (5 yr DFS). This figure is used because results of 
treatment are very frequently reported in the medical literature in this form. In addition, in HD, very few 
patients relapse after this period, and the 5 yr DFS can be used as an index of cure. 

ii) Morbidity Units - in addition to its mortal risk, each diagnostic test and each treatment has an associated 
risk of morbidity. This includes both the risk of severe complications, such as pulmonary embolism after 
staging laparotomy, and the duration of pain and incapacity associated with the test or treatment under 
normal circumstances. 

For each test and treatment we asked a number of physicians, directly involved in the investigation and 
treatment of patients with HD, for their estimate of the "pain and discomfort", and the "duration of 
incapacity", for the "average" patient undergoing a given test or treatment. An arbitrary standard was made 
which defined the "pain and discomfort" of a bone marrow biopsy or of a single day's incapacity each 
equivalent to 10 units. The literature was then searched for data on the complication rates of tests and 
treatments. Each complication was assessed according to the same criteria as the basic test; this figure, 
ltiplied by the incidence of the complication is included in the morbidity values shown in Table 3.2. 



mu 



iii) Dollar Cost of Tests and Treatments. These are shown in Table 3.3. 

Each of these types of utilities is evaluated separately. At present, we have not found a satisfactory way to 
incorporate these disparate elements in the same scale of values, and no attempt has been made to "trade-off' 
one type of utility with another. 
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Table 3.2. Morbidity Units for the Staging Procedures and Treatments 



Gallium scan 

Bone marrow biopsy 

Liver biopsy 

Lymphangiography 

Peritoneoscopy 

Laparotomy 

Extended mantle radiotherapy 

Total nodal irradiation 

MOPP chemotherapy 


7 units 
10 units 
25 units 
40 units 
80 units 
875 units 
1000 units 
1696 units 
3000 units 




Table 3.3. Dollar Cost of Tests Used 


in Staging Hodgkins Disease 






Test 
Cost 


Professional 
Fee 


Total 


Liver-Spleen Scan 


213 




60 


273 


CT Scan 

- with contrast 

- without contrast 


168 
118 




100 
70 


268 
188 


Gallium Scan 


259 




60 


319 


Liver Biopsy 


72 




155 


900* 


Bone Marrow Biopsy 


94 




80 


154 


Lymph angiogram 


421 




276 


696 


Laparotomy 








6000-8006** 



* includes 2 days 1n hospital, type and cross-match, etc. 

** this includes 7-10 days In hospital , fees for surgeon, physician, 

and anesthesiologist, use of operating and recovery 

room, laboratory tests, X-ray*, and pathology fee-s for operative 

specimens. 
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3.4 Evaluating Decision Branches 

This step is known as "averaging out and folding back", or in more technical terms "the process of backwards 
induction in the theory of dynamic programming". Starting with the most peripheral decision nodes, all 
outcomes of a particular "sub^decision" are evaluated* and the option with the highest utility ascertained; 
this value is then assigned to die decision nod?. For example, in the decision tree shown below, one would 
initially evaluate each of three branches after lymphangiogram. 



BMBX 



■6 



Rx 



o 



Rx 



LAG 



-6 



MOPP 



Rx 



/ 



+ ' 






LAP 




Rx 


Equiv 


LAP 




Rx 


— 






LAP 



To evaluate these, one must know the probabilities of pathologic stage at that particular point in the tree, 
when the prior probabilities have been modified by a negative BMOX result and by the result of 
lymphangiogram. We will use the symptomatic male patient from the example in Section 3.2. We will 
evaluate the branch indicated by the arrow: me probabilities of pathologic stage for this patient at this 



December 1981 



■31- 



Iloilgkins Decision Analysis 



decision node, with negative bone marrow and positive lymphangiogram are: 



Stage I+II: 


0.22 


Stage III: 


0.64 


Stage IV: 


0.15 



When die patient's stage is not known for certain, assessing the utility of die option to proceed directly witii 
treatment, involves calculating die average disease-free survival for each treatment, and choosing the 
treatment with the highest value. The utility for a pardctfar treatment is calculated as a weighted average, 
using the probabilities of stage and the five year disease free survival values for each stage for that treatment, 
e.g. assessing the value of total nodal irradiation for the patient: 



Stage Probability of 5-year DFS 

that stage for that stage 

with mi 

I+IIB 0.22 times 70% » 15. 4% 

IIIB 0.64 tines 25% ■ 16.6% 

IVB 0.16 tines OX » 0.0% 



Weighted average = 31.4% DFS 

This is then compared with the weighted average obtained from die corresponding values for MOPP 
chemotherapy: 



Stage Probability of 5-year DFS 

that stage for that stage 

with NQPP 

I+IIB 0.22 times 60% * 13.2% 
IIIB 0.64 times 50% = 32.0% 
IVB 0.15 times 35% = 6.25% 



Weighted average = 50.45% DFS 
In this example the weighted average for MOPP chemotherapy is greater man that for total nodal irradiation, 
and would thus be preferred to TNI as the treatment option. The same techniques are used to evaluate 
combined modality therapy as one of the treatment options, but the calculations arc more complicated 
because, as already described, one must aHow for salvage of patients who initially receive single modality 
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therapy. 

To assess the value of laparotomy, the probability of each stage, is multiplied by the best Treatment for that 
stage, e.g. to assess the utility of laparotomy before any other tests have been performed, again a weighted 
average is used as follows: 

Stage Probability of 5-year DFS 

that stage for that stage 
with BEST Rx 

I+IIB 0.22 times 70% TNI * 15. 4% 
IIIB 0.64 times 50% MQPP -32.0% 
IVB 0.15 times 35% MOPP «. 5.25% 

Weighted average ■ 52.65% 

To obtain the true utility for laparotomy, one must also take into account its mortality risk - this is usually 

assessed as 1%, which includes 0.5% peri-operative mortality risk, and a farther 05% late mortality risk from 

post-splenectomy sepsis. The laparotomy branch of the decision tree actually looks like: 



Laparotomy mortality 



LAP 



p-OOl 



t 



l + IIB 



O 



Survive surgery 



p = 0.99 



■6 



HtB 



IVB 



The value obtained in the calculations above for the average utility of LAP must therefore be multiplied by 
0.99 to. give the actual value which should be compared with the best treatment option (52.65% X 0.99 = 
52.12%). When this modified value is compared with that derived for immediate treauncnt prior to any 
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testing, it is apparent that the laparotomy option (52.1%) still gives a higher chance on the average for 
disease-fiee-survival than the preferred treatment option MORP (50.45%^ so its value is assigned as the value 
for mat decision node. 

The other decision nodes which follow lymphangiogram are evaluated similarly, the only difference being the 
probabilities of stage which occur with equivocal and negative rymphangiograms. 



Positive 52.1% DFS (LAP) 



LAG 1 Equivocal 6 1 .7% DFS (LAP) 



***&&* 63.8% DFS (LAP) 



To determine the value of the whole lymphangiogram branch - or the value of any branches after a chance 
node - one must first calculate the likelihood of each outcome after the chance node, and then multiply the 
expected utility of each branch by its likelihood This gives the overall expected utility for the whole 
lymphangiogram branch. 



Positive 



0.42 



0.18 
Negative 



0.40 



52.1% (LAP) 



LAG i. Equivocal 

Q— zzz 6 1 -7% (LAP) 



63.8% (LAP) 



Expected DFS for lymphangiogram branch 



21.9 



11.1 



25.5 



58.5% 



Calculations for the whole tree arc carried out similarly: 



December 1981 



■34- 



Hodgkins Decision Analysis 



Rx 
[53.8, M0PP1 



BMBX 



p = .93 
.51:.40:.09 



■n 



[56.9] 



■O 



LAG 



[58.5] 



p - .07 
0.0:0.0:1.0 



Rx 



[52.6] 

When all the calculations are performed, the utility of the BMBX option is 56.9% DFS, which exceeds that of 
immediate therapy (52.6% DFS). , 

The best course of action in this situation is therefore: 

To perform bone marrow biopsy; if positive give MOPP, if negative further testing is 
needed. Regardless of the results of lymphangiogram, laparotomy has a higher utility 
than the treatment options. This indicates that lymphangiogram is not necessary for 
this patient, but that laparotomy is needed to select appropriate treatment 

In summary, any decision tree is evaluated in the following way. Starting with the "leafiest" part of the tree, 
i.e. that furthest away from the fundamental decision, utilities are calculated in two ways: 
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1. at a decision node utilities are calculated for each branch and the highest one is selected as the 
value of the node. 

1 at a chance node utilities are calculated for each subsequent branch; the utility of a particular 
branch is then multiplied by its probability and a// the products obtained are added together to give 
a weighted average which is the expected utility for that chance node. 

In a decision tree as complicated as that for HD, where ftere are more than 2000 decision branches to 

evaluate, it fa necessary to use a computer for the calculations. 
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4. Conditional Independence Assumptions 

For each patient with Hodgkins disease, we start out ..with information about histology, symptoms, sex and 
age. As further clinical information is considered, and diagnostic tests are performed more clues about the 
spread of disease are obtained. In the decision analysis program we repeatedly ask the question: given the 
information available so far, what are the probabilities of eicb stage in this patient? 

Suppose we have a large number of past patients whose stages are now known, and who are identical to the 
current patient in all relevant respects. Probabilities about the new patient's stage could be estimated directly 
from the actual distribution of stage among the earlier patients. Thus, for example, if 30% of the former 
patients were in stage III, 30% is a sensible estimate for the probability of stage III disease for the patient at 
hand. 

As more and more findings are added to the database, however; theeohert of past patients highly simitor to 
the present one becomes smaller. Probabilistic predictions obtained under the method just described would 
be subject to such large sampling error as to be almost useless. If our calculations of staging probabilities are 
to be at all informative, certain independence assumptions must be fairly accurate. These assumptions, 
crudely speaking, assert that the way a particular finding is used in lrfcdffymg a patiehfs prognosis does not 
depend on any other toformation about the patient 

In this section, we discuss our initial assumptions of independence, the tests we performed to validate them, 
and a modification of the program that arose when one of the independence assumptions was found 
inconsistent with our data. 
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41 The Independence Hypothesis 

Suppose that, for a given patient, we have a set Y of prior findings and a new finding C. Bayes* Theorem 
asserts that the probability die patient is in stage i = 1,2,3 or 4) given this information fofiows: 



P(Y|i) X P(CH and Y) 
P(1|Y and C) — ---ii-Jtiiw-u-j^^*.-.-— ■• x P(i) (1) 

P(Y and C) 
where fl(i) = probabSky of % PflY) = conditional prooaMity of i given Y. 



The independence assumption we make is that P(C|i and Y) = P(Q), which means that, among those in 
stage i mere is no correlation between those with finding C and those with finding Y. Under this 
independence assumption, Equation (1) becomes: 

P(YJ1) X P(C|1) 
P(tfY and C) ■— -* - — **-r-r B -K- Jt W*l O*} 

P(Y and C) 

The value using equation (1A) instead of (1) is that is allows us to use larger data see, less subject to sampling 

error, in the calculation of needed probabilities. 

Of course, if the independence assumption is false, calculations based on it cannot be trusted. Thus it was 
necessary to perform statistical tests of the null hypothesis Hq that, F^CH and T$ .sf B(C|i) where C and Y went 
varied The tests we employed were tied to the familiar Chi-squared test 

4.2 Tests of 

Fw all patients with Hodgkins disease we start witii information about histology, symptoms, sex and age. 
Our initial estimates of staging probabilities were based on histology and symptoms alone. Thus, for example, 
the data-based estimate of the probability of being in stage III given mixed ccHutarity histology and die 
presence of symptoms <e.g. night sweats) was 0.30. 
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The data make clear that the very fact of being male makes one more likely to be in a later stage. Probabilities 
of stage given sex as well as symptoms and histology were estimated by the program under (1 A), under the 
assumption that P(male given i) = P(male given i and Y), where Y is ; any one of the eight possible 
combinations of findings about histology and symptoms. The number 8 arises because there are four possible 
histologic subtypes and key symptoms are either present (B) or absent (A). 

The following example will show how we used the Chi-squared test to examine the null hypothesis of 
independence among the basic findings sex, symptoms and the histological subtype (Eisen, 1977). In our 
database, 51% of the patients in stage I are male. 164 of the patients are asymptomatic (A), of whom 132 have 
the Nodular sclerosis (NS), 17 have Mixed cellularity (MC), and 15 have the JLymphocyte predominant (LP) 
histological subtypes. Under the independence hypothesis, the number of males in stage 1 who are also NS 
and A would be expected to be 132 X0.51 = 67. The expected number of males whd are A and MC, and A 
and LP, would be 9 and 8 respectively. Consider the table befow: 

Stage I Asymptomatic Patients 

Histology Expected number of Males (Ej) Actual number 

Under Independence Hypothesis of Males (Aj) 

MS (132 patients) 87 6° 

MC (17 " ) 9 11 

LP (16 ) 8 13 

The viability of the null hypothesis depends on whether the observed differences between the expected (Ej) 
and the actual (A:) numbers of males can reasonably be dismissed as chance fluctuations. We calculate the 
"discrepancy index" Bj under the rule Bj = (Ej - Aj) /"Ej for each of the categories above. We make similar 
calculations for those males who did have symptoms, and also for the females in stage I. The sum of the Bj's 
for all categories is compared to some cutoff number Z, with the null hypothesis rejected if the sum exceeds Z 
and accepted if the sum is at or below Z. Z is dependent on the number of different categories, and the extent 
to which certain categories arc "redundant" of others; both these considerations are very familiar to users of 
Chi-squarcd tests. 
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Ii turns out that, for sex vs. histology and symptoms, the independence assumption was not acceptable under 
the statistical test Thus to indude the finding "sex" in the prediction, we had to go back to Equation (1) 
rather than use die simpler form (la). We then proceeded in similar fashion to see whether age could be 
treated as independent of sex, symptoms, and histology in the estimation of staging probabilities. Here, me 
independence assumption passed its test easuy. 

We also investigated whether the errors hi various diagrnsstk tests couM rje viewed as independent events. 
(Eg. does a fake positive result in a Gallium scan increase ti# probability of a smuTai' error in a 
lymphangiogran?) The data did not compel us to reject till independence assumptioni but often we had too 
few data points to allow any definitive statements. 

In summary, we did not forget mat our Bayesian procedure for revising staging probabilities was based on a 
strong independence assumption. Some tests of the assumption's validity were in order and were performed. 
Except in one case (sex vs. histology and symptoms), the tests indicated that the assumption was close enough 
to the truth mat using it did not introduce serious errors in the calculations. In the case just mentioned, where 
the independence assumptions failed 1 , we recatoilatf^ probabilities A less direct method of testing our 
assumptions was the full-scale evaluation of the program as described hi Chapter 5. Th» evaluation showed 
that our predictions of stage, even when many findings were used by applying Bayes' Rule repeatedly, were 
reasonably accurate. This would not have ocoirred had our assum 

We feel that our procedure, including the tests of independence, came as dose as possible to avoiding me twin 
evils of biased predictions and predictions with very high variance, either of which would have doomed our 
efforts. 



1. One might wonder whether, when many tests of independence are performed, chance alone would lead one of mem to yield a 
negative result. In this situation, the rejection was sufficiently emphatic that we did not think this was the case. 
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5. Testing the Bayesian Model for Predicting Pathological Stage 

In Chapter 4, we showed that the independence assumptions that underlie the Bayesian calculations of staging 
properties were consistent with the data used to calibrate our computer program. But, there is another issue 
that must be addressed before the program can be considered for general use. Can a model based largely on 
the experiences of patients from large metropolitan teaching hospitals reliably be used in making prognoses at 
other hospitals? 

There are several reasons one might fear this question should be answered in the negative. More than half the 
patients in the database came from a well-known radiotherapy refierraleenter. which is perhaps more likely 
than other centers to attract patients with localised disease. This circumstance raises the possibility that 
probabilistic predictions based on data dominated by these patients, might be overly optimistic for patients 
elsewhere. Furthermore, the data base consists solely of laparotomy-staged patients. This could lead to 
underestimation of the prevalence of stage IV disease, for some patients in this stage could have had their 
extra-nodal involvement confirmed without laparotomy. Moreover, the predictions depend on the histologic 
subtypes reported for the patients. Pathologists tend to differ somewhat in their assessments of histologic 
subtype (Jones et al, 1977); to the extent that pathologists differ from their counterparts elsewhere, the 
"transferability" of results is diminished. 

Finally, each of the prior and conditional probabilities used in the program has an uncertainty due to 
sampling error, an uncertainty which may be increased by the repeated use of Bayes Rule. In principle, we 
could estimate the uncertainty in our final staging probabilities as a function of the uncertainties in the 
numbers that determine it. However, the calculations would be very difficult in practice, and ultimately of 
quite limited value if we could demonstrate that the program makes accurate predictions of stage. 

The considerations suggest the importance of testing the general predictive power of the decision analysis 
program. In this section, we describe 2 separate empirical tests, one conducted in 1977 and the other in 1980. 
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Both tests, as we will see, suggest that die program's predictions are accurate to within an irreducible 
margin-of-error caused by usual chance fluctuations. 

5.1 The»77 Test 

The test population we used consisted of 156 Hodgkins disease patients, 82 of whom were treated at Center A 
and 74 of whom were treated at the Tufts-New England Medical Center Hospital. We thought it appropriate 
to consider the two groups of patients separately, so that interesting, but opposite patterns, if they arose, 
would not cancel one anotiier out For each patient teft*matibti wis chained about histology, 
symptomatology, age, sex, presenting lymph nodes, spleen ant 'fiver scan re^^^teaftalme phosphatase 
levels 1 . These data were fed into bur program 

patient's stage under die Ann Arbor criteria. To as^M ai&iracy of die program, we compared these 
probabilistic predictions with tire patient's actual stage as determmed by laparotomy. 

An obvious question arises: how does one tell if a probabilistic prediction is accurate? Suppose, for example, 
the program predicts that patient A has a 60% chance of being in stage I or II and a 40% chance of being in 
stage III or IV. Even if die patient is found through laparotomy to be in stage HI, the prediction is not really 
wrong, for it had indicated considerable uncertainty about how far the disease had spread. But, while it is 
difficult to talk meaningfully about the accuracy of a given patient's prediction, it is somewhat easier to do so, 
given a large set of simitar patients. 

Suppose, for instance, that there are SO patients for whom the program makes die 60%/40% prediction just 
mentioned If these predictions are all accurate, one would expect that, except for random fluctuations, die 
number of patients who actually emerge in stages I and II would be SO X .6 = 30. Thus, if die number 
observed in the two early stages is significantly greater than 30, this means the program was too pessimistic for 



1. Not aU information was available for every patient 
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these patients. If, on the other hand, the number is far below 30, the program was too optimistic. 

Our statistical test of the program's accuracy for the two gnnurof patients :% based on the principles cited 
above. In performing the test, we combined stages I +11 because, for clinical purposes, the two stages are 
virtually the same. We also combined stages III and IV because, given the- small test populations, we expected 
so few stage IV patients that we will be unable to make meaningful statements about the stage IV predictions. 
For each patient in the test, we recorded that program's estimate of "r", the probability the patient is in stage 
I+IL The probability that the patient was in stage III or IV is simply fl-rj. We then ordered the patients 
according to their r-values, from highest to lowest 

According to the criteria discussed in Appendix II, we broke the patients into groups within which the r 
values are close together. Finally, we compared the expected number of stage I +11 patients in each group 
with the actual number as learned from laparotomy! We present some of out results in Tables 5.1-5.4: 



Table 5.1. 


Evaluating die TUFTS patients, using Hfetotegy,^jrBipt6ins and Sex 




Group 


Number 

in 
Group 


Average 
probability 
in group 


No. in Stages I+II 
Expected Actual 


W-Statistic 


1 
2 
3 


22 

26 
26 


0.71 
0.67 
0.3 


16.64 

14.90 

7.72 


16 

17 

8 


-0.26 

0.84 

0.12 




74 




38.17 
Z ■ 0.79 


40 





In the tables the W-statistic for any group is a measure of wheflier the discrepancy between expected and 
actual outcomes can reasonably be attributed to chance alone. The statistic has only a 5% chance of exceeding 
2 in absolute value if all predictions in the corresponding group arc correct. Under the null hypothesis that all 
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Table 5.1 Evaluating tbe Center A patients, using Histoiosy, Symptoms arf Sex 
Group Number Average Ho. f» Stages I+II 

in probability 

Group f* group Expected /Ictus 1 



M-Statistic 



1 


16 


0.7 


11.24 


12 


6.41 


2 


28 


©.6 


16.8* 


18 


8.84 


3 


21 


0.44 


8.26 


8 


-0.63 


4 


22 


0,31 


6.73 


8 


0.66 



1.64 



Table 5.3. Evaluating the TUFTS Patients, using Histology, Symptoms, Sex, Age and Alkaline Phosphatase Levels 
Group Number Average No. 1n Stages J>ll W~Stati at ic 

in probability 

Group in group Expected Actual 



1 


23 


0.74 


16.91 


16 


-0.81 


2 


14 


6.61 


8.62 


10 


0.68 


3 


14 


0.6 


7.06 


8 


0.26 


4 


21 


6,28 


6.81 


6 


-6.41 



1.89 



Table 5.4. Evaluating the Center A Patients ii^Htstology,SjBipt«ns, Sex, Age and Left-Neck Nodes 
Group Number Average No. in Stages 1+ II W-Statistic 

in probability 

Group in group Expected Actual 



1 


17 


0.76 


12.74 


14 


0.7 


2 


16 


0.67 


10.1 


8 


-1.16 


3 


18 


0.67 


10.28 


13 


1.30 


4 


16 


0.47 


7.08 


6 


-0.68 


6 


22 


0.32 


7.03 


6 


-0.48 



3.60 



predictions in all groups arc correct, the statistic Z obtained by summing the squares of the W-values should 
foBow a Chi-squarcd probability distribution, with as many degrees of freedom as there are groups. Thus at 
the usual 5% level, one would reject die null hypothesis only if Z exceeds 7.81 when there are three groups, 
9.49 when there are 4 and 11.07 when there are 5. Tables 5.1 and 52 indicate mat the projected distributions 
on stage, based on histology, symptoms and sex were accurate at both the New Fngland Medical Center 
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Hospital and Center A. Tables 5.3 and 5.4 suggest that the initial probabilities, modified in Bayesian fashion 
as additional findings become available, are also accurate. Revised predictions for the subset of patients who 
also had liver and spleen scans were as consistent with the data as those summarized in the tables. These 
tables imply that our concern that the predictions largely derived from a radiotherapy referral center would 
not be relevant elsewhere has not been realized. 

At this point the reader might be wondering why, if the initial predictions in Tables 5.1 and 5.2 were accurate, 
we bothered to revise them. The reason is that we need predictions mat are not only accurate, but as helpful 
as possible. To a clinician deciding between treatments it is far more informative to know that the probability 
of a particular contingency is 0.99 rather than 0.50. Since including new findings in the predictions tend to 
polarize them towards 1 (certainty) and zero (impossibility), the revised predictions are generally somewhat 
less equivocal than the early ones. 

Once the Tufts and Center A patients were shown to be compatible with die original database, it seemed 
appropriate to include their data in an expanded data base. The program that resulted from merging the data 
was based on 900 rather than 700 patients. Of course the very compatibility of the new patients with the 
original ones means that the merger brought only slight changes to the program. 

5.2 The 19» Test 

In 1980, we decided to perform another test of the validity of the program outside the hospitals involved in its 
calibration. The reasons for mis new test were that (i) the 1977 data samples were too small to allow the 
consideration of stage IV predictions in their own right, and (ii) the problems inherent in the variable 
assessments of histology may have been artificially reduced in me 1977 study because the test Hospitals were 
similar in terms dfrtistopathology. 

'ITic 1980 test population consisted of all new pathologically-staged cases of Hodgkitts disease arising in 100 
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cities and towns in Eastern Massachusetts over a defined period of time. Confidentiality of data was 
maintained, witii berth patients and hospitals indentified hy study numbers only. The original database was 
searched to insure mat none of mis test population was included in me existing database. Cluneal 
information about these patients came from 70 hospitals in the study area; in aB, data on histology* sex, 
symptoms, age, and sites of lymphatic involvement was available for 301 pathologically-staged patients, 206 of 
whom had lymphangiogram tests. 

The method for testing accuracy was similar to that used hi 1977, except mat we wanted to learn about the 
reliability of all three of the predictions of stage rather man just the two groupings used previously. The test 
procedure, which is described in detail in Appendix H, involved choosing at random one of die three 
predictions made for each patient We called the stage to which the chosen prediction for a given patient 
referred, the "stage of interest". Thus, given 361 patients, we were really assessing the accuracy of 100 
predictions about stage I+H, WO about HI and 100 about IV, 

Tables 5.5 and 5.6 summarizing our test results appear below. The W-statistics can be interpreted the same 
way as those that appeared hi tables 5.1-5.4. The hypothesis mat all predictions are accurate is statistically 
unacceptable at the 5% level if Z is greater than 1652 for 9 groups, and 18 Jl for 10. 

Tables 5.5 and 5.6 make clear mat die program's predictions are accurate to within the level of chance 
fluctuations. Thus, once again, we can have confidence in die original probabilities assigned to various stages, 
and in the somewhat modified estimates that arise when additional information (generally including 
lymphangiogram results) is taken into account 

In summary, our tests in 1977 and 1980 offered strong evidence that the various potential problems in our 
Baycsian program are of little practical importance. The results increase our confidence that the program can 
be useful in a wide variety of hospitals. 
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Table 5.5. 1980 Test of predictions based on symptoms, sex and histologic subtype 

Number of staged patients with this information available: 301 
Actual number overall in "stage of interest": 99 
Expected number overall fn> stage of interest":' 100.9 



Group 


Number 


Expected 


Actual 


1#-Stat1*tic 




in 


in "stage 


in 


"stage 






Group 


of interest" 


Of 


interest" 




1 


37 


24.94 




28 


1.21 


2 


20 


12.84 




11 


-0.70 


3 


11 


5.92 




5 


-0.06 


4 


11 


5.47 




3 


-1.42 


5 


14 


6.73 




9 


1.08 


6 


12 


5.66 




4 


-1.08 


7 


13 


5.91 




5 


-0.78 


8 


19 


7.51 




6 


-0.60 


9 


34 


11.07 




9 


-0.78 


10 


17 


5.18 




5 


-0.27 


11 


113 


9.73 

Z = 10 


.39 


14 


1.60 



Table 5.6. 1980 Test of predictions using clinical findings and lymphangiogram results 

Number of staged patients with this information available: 206 
Actual number overall in "stage tot interest": 83 
Expected number overall in "stage of interest": 66.48 



Group 


Number 


Ex 


pected 


Actual 


W-Statistic 




in 


in 


"stage 


in 


"stage 






Group 


of 


interest" 


of 


interest" 




1 


28 




22.80 




24 


0.73 


2 


17 




11.87 




9 


-1.44 


3 


12 




6.88 




7 


0.18 


4 


11 




5.04 




4 


-0.83 


5 


14 




5.05 




6 


0.06 


6 


18 




5.02 




3 


-1.02 


7 


106 




9.81 




11 


0.63 



Z - 4.68 



December 1981 



-47- 



HoiJgHns Decision Analysis 



6. Ewwple of a Typical Patio* Sestoa 

We will demonstrate the program with a typical patient session. The patient is a 25 year old male who has 
dropped 20tb in weight from his usual 170ft. fie has no othergpnaraliTr d symptom* and has sought medical 
attention because of a slowly-increasing mass on the felt side of his neck, Biopsy of this left-neck mass 
revealed Hodgkins dfcease of the nodular sclerosis histologic subtype. Investigations performed at this time 
include a chest X-ray and a liver-spleen scan, bom of which pere normal. We win use this basic information 
to predict the likelihood of each of the stages of the disease, and use this knowledge to plan further 
investigations tailored to treatment 

First the program asks the three question which give inrbrmauoh relating to the three interdependent findings 
from which the prior probabilities are estimated. In the dialogue between computer and user, the computer's 
questions are in th 1 s typeface, the user's responses in Ualk typeface. 

Is the patient A (asymptomatic) or B (.symptomatic.)? B 

What 1s the histologic subtype of the patient? Nodular sclerosis 

What is the sex of the patient? Mate 

The program men refers to its database to give the appropriate prior probabilities, ie. 



Stages I+II 


0.34 


Stage III 


0.60 


Stage IV 


0.18 



Having established these, further questions are asked to characterize the patient With each question the 
probabilities are modified by Bayes Rule, using the conditional probabilities of finding given stage. 

In what age group is the patient? 12 to 39 years 

This is the correct answer for this patient but by pressing T all possible answers the program wQl accept are 

displayed, Le. 
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Unknown 

Younger than 12 years 

12 to 39 years 

Older than 39 years 

•Display data about fact... 

•What if 

If the response "*WHAT IF ..." is selected, the program oUsplays^sposjerioiiprobabiMtjes ft* each possible 
answer. 

Younger than 12 years: I+II 0.21 III 0.52 IV 0.27 

12 to 39 years: I+II 0.39 fll 0.51 IV 0.10 

Older than 39 years: I+II 0.20 III 0.48 IV 0.32 

The "*DISPLAY DATA ABOUT FACT" option displays the data known about the relationship between the 
patient's age and the stage of the disease, and the conditional probabilities of age given stage, by which the 
prior probabilities are modified 

Younger than 12 years I+II 0.03 III 0.06 IV 0.08 
12 to 39 years I+II 0.84 III 0.74 IV 0.45 

Older than 39 years I+II 0.14 III 0.23 IV 0.47 

After one enters the appropriate age group for this patient - 12 TO 39 YEARS -* tiie prior probabilities are 
modified by Bayes Rule, using these conditional probabilities, in mis case altering them in favor of early stage 
disease. 



Stages I+II 


0.39 


Stage III 


0.61 


Stage IV 


0.10 



The user is then asked consecutively about left neck/right neck and mediastinal nodes, which make modest 
changes in the probabilities. Again the same explanatory features are available. 



Were there any nodes detected on the left 

supraclavicular or cervical areas? Yes 



Stages I+II 


0.37 


Stage III 


0.53 


Stage IV 


0.10 
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Were there any nodes detected mi the rtglit 
supraclavicular or cervical areas? 



0.M 
t.§4 
0.0* 



No 



1*11 
III 
Stage IV 



t? 



M> 



la there medfaatfaal favel 

Stages I+II 0.39 
Stags III 0.88 
Stage IV 0.07 

Next, information is asked sefwatriy about ^,?p^r scan and the fiver scan. Once Again the suae 

explanatory features are available. Although these fever-spleen scan results were obtained simultaneously, 

because of their different interpretation, questions about tiidr results are asked separately. 



ha 



What is the appearance of the spleen on scan? 
Acceptable answers Ihchide: 



Momtai 



Normal scan 
Enlarged |> IS cm) 
HI 1 infl defects 
Both enlargement 




**can 

scan o< ■.■■< ■'■: \-j- ,- 

IHng defects seen on scan 



With Normal scan the probabilities are modified to: 



Stages I+II 


0.48 


Stage III 


0.48 


Stage IV 


0.08 






tft&MBCt J9Rr 



■»-. 



thdtkka Dechkm Jta ftjft 



Was the uptake pattern of liver scan normal or abnormal? 



Possible answers: 






Normal 








Abnormal 








Unknown 








•What if... 








•Display data 


about 


fact. . . 


Stages I+II 




0. 


49 


Stage III 




0. 


49 


Stage IV 




0. 


03 



Normal 



The false positive and negative rates for the liver/spleen scan are given m Table 1.2. They have been 
combined with information from the database regarding the likelihood of an involved liver or spleen in the 
various stages to give conditional probabilities for the the probability of a particular scan result, given the 



Having used this basic information to characterize the patient, and obtained the above "posterior" 
probabilities of stage, the program now enquires which, if any, of the more invasive tests have been 
performed. For any mat have, probabilities are further modified according to Bayes Rule, using false positive 
and false negative rates that have been derived from the literature, again modified where necessary to link 
results of the tests directly to pathologic stage as in the example shown for Gallium scan in Section 2.2.2. 

The tests asked about at this stage include bone marrow biopsy (BMBX), percutaneous liver biopsy (LBX) or 
peritoneoscopy with guided hver biopsy (PTX), gallium scan of abdominal nodes (GALL) lymphangiogram 
(LAG), and staging laparotomy (LAP). 

The user is men asked whether any of these test procedures arecontraindicated. For each procedure the user 
is prompted with a list of the recognized contraindications ofthe test, e.g. for lymphangiogram: 
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Boss th« patient have any s# tha f off ovf n§ coat rai nd feat ions for 



Prwrfaws rarffotlwrafy to 
Vascnfar starts 
lodida Ityasrssssittvtty/ 
Rospiratery iMsainaaat 
Aetiva thro— opMnSUH 

Any procedure whkfa is ceo t ra ii i ^ ca te di 



Decision analysis is mm used to approach the TEST verso* TREAT decision, ie. whether it is necessary at 
this stage to perform some or all of the more invasive tests, or whether treatment can be gjv«n without more 
ado. TTie option of usmg combined modality therapy is aho otftwA The tuer is reminded feat, when 
comparing 5 year disease-free survival <DFS> values after Sns MWKff arat ater <tnek modatity theapy, 
allowance should be made ibr a second remission wirh chemotherapy if relapse occurs after single inodamy 
therapy. If the user dects to use coinbined modality therapy it is added to tlwse considered in the TREAT 
branches of die derision tree; for symptomatic patients, such as me oae under consideration, these 
treatments would then include combined modality therapy (CMX total nodal irradiation (THE), and MOPP 



As all the possn^ biancbe*of 6* aa^ fonn,as 

below, with die first step of the plan in the left hand eoharm. 



First step 


Avar ago 


Avg Morbidity 


Avg Morbidity 


Dollar c 


in p^lao 


6-yoar i)f $ 


,*fv:taa*1ajiii SE *\ 


of txaatawftt 


,0*;al**.: 


Total nodal 


0.5956 


0.6 


l*W 


3560 


HOOT 


0.6742 


6.6 


iftgff * 

vvft 


ma 


CM 


0.6788 


6.6 


488w 


4266 


Laparotomy 


0.7366 


888 


3729 


16666 


Gstltast scan 


6.7189 


' 876 ' . . • - 


'•^fWasv- 


f688* 


Lynphanglograa 


0.7288 


988 


3728 


16688 


Bona narrow 


0.7367 


888 


xMa*.-, 


18688 


Livar biopsy 


6.7386 


86t 


3728 


t892w 



First, note that the plans are listed according to dwir first step. If das is a treatment, men there are, of course, 
no further steps. If the first step is a test, die utilities shown are calculated for the entire Plan, and include any 



IHumhirfMt 



■M- 



/frrifrftfrr ftvnfr* ^wfttfr 



subsequent tests and treatment For example, for the plan beginning with bone marrow biopsy, the plan with 
the highest utility, the whole plan looks like this: 



BMBX 



I + 11 



Negative » = > LAP 



p - 0.49 



p - 0.99 



6 



Positive 



p • 0.01 



■O 



MOPP 



111 + IV 



p ■ 0.51 



Combined modality 



MOPP 



This plan shows that when bone marrow biopsy is performed, there is a OjOI chance of this test being positive; 
this chance is low because the probability of stage IV in this particular patient was Only 0.02. Since a positive 
bone marrow biopsy is conclusive proof of stage IV disease, MOPP can then be given without any further 
testing. Ninety-nine percent of the time the bone marrow biopsy will prove negative. At this point the whole 
TEST versus TREAT decision is approached again. The best step next, with the probabilities revised by the 
negative bone marrow biopsy, is to perform a laparotomy, with the chances of I+IIB disease being .49, and 
III & IV being 0.51. Once laparotomy has been performed, stages are known for certain, and no further 
decision is necessary. 

At mis point the user is given the option of displaying a number of other items including: 
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All pi an s - these are displayed as decision trees similar to that above 

One p 1 an - any individual plan can be displayed by designating its first step 

Summary of pi ans - very similar to the previous tabular display except that plans are displayed 
ranked according to their DFS utility. 

Verbose version of one plan - a particular plan, specified by its first step, is described in 
words. 

Patient summary- all the clinical and test data known about the patient are displayed. 

Following this DISPLAY option, the user is presented with the further options which make the program so 
flexible that its use can be adapted to a particular patient, or to the experience of any hospital. These options 
include: 

Change patient information - Jurther specified as: 

Clinical information 
Test information 

Modify information about tests or treatments - Jurther specified as: 

Prior probabilities 

False positive/negative rates for questions or tests 
Laparotomy as a perfect test -can assign a FN rate 
Mortalities for tests - the rales used are displayed together 

with an option to change them 
Morbidities for tests - see Table 3.1 
Morbidities for treatments - see Table 3.2. 
Survival data for treatments - these can be changed according 

to institutional experience. 
Consider a different treatment 
Delete a treatment from consideration 

Reanalyze current patient - if any of the MODIFY options are selected 
one must use this option to recalculate plans. 

Supply your own plan - here the user is given the opportunity to supply 
his/her own plan for management, and compare it with those 
calculated by the computer. 

We will try some of these options for our patient Imagine that there has been some doubt about his exact 
histologic subtype, and the pathologist notes it has some of the features of the mixed cellularity subtype. We 
want to see what difference this will make to the probabilities of stage, and hence lo the management plans. 
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What do you want to do? Change patient information 

What do you want to change? Clinical information 

The clinical information known about the patient will then be displayed, and the item to be changed 
designated. In this case, changing the histologic subtype from nodular sclerosis to mixed cellularity changes 
the probabilities from I +11 0.49 : III 0.49 : IV 0.03 to I +11 0.37 : III 0.55 : IV 0.08, i.e. it increases the 
probability of more advanced disease. 

Further, supposing you are not very happy about the patient's lung function -- he is prone to asthmatic 
attacks, which you feel may affect the mortality risk for both the lymphangiogram and staging laparotomy. 

What do you want to do? Modify information about tests or 

treatments 

What do you want to modify? Mortality rales for tests 

Which test? Lymphangiogram 

The current mortality for lymphangiogram is 0.001. 
What would you like to change it to? 0.005 

Which test? Laparotomy 

The current mortality for laparotomy is 0.01. 
What would you like to change it to? 0.02 

Also you wish to consider as an alternative treatment for this patient, the "sandwich" form of combined 
modality therapy, in which 3 courses of MOPP arc given, then a course of radiotherapy, then a second 3 
courses of radiotherapy. Since this option replaces the more conventional combined modality therapy, it is 
first advisable to delete combined modality therapy from consideration, before indicating you wish to 
consider a different treatment. You will be asked at this point for the name of the new treatment and for 
appropriate disease free survival and morbidity data. 

What would you like to do? Consider a different treatment. 

Type the name of this treatment "Sandwich" 

Five year disease free survival for "Sandwich" given I+IIB: 0.75 
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Five year disease free survival for "Sandwich" given III8: 
Five year disease free survival for "Sandwich" given I VB: 

What is the expected morbidity associated with Sandwich 

What would you like to do? Reanalyze current patient 



0.4 
0.35 

3500 



Now, the program again asks whether any invasive tests have been done already, or are contraindicated. It 
also asks whether combined modality therapy is indicated. Although standard combined modality therapy 
was deleted from the program, it was replaced by the "sandwich" combined therapy so we need to answer 
YES to this question so that salvage will be considered after the single modality therapies. 

The computer then calculates a new series of plans, which incorporate the changes just made, i.e. the different 
prior probabilities, the increased mortality rate fi>r the lymphangiogram and laparotomy, and the new 
"sandwich" treatment 



Wf §' mo r% ^1 #f ty -flail er cost 
Mitt ''' of plan 



3560 
1000 

8628 
8814 
6048 
8473 
6019 



First step 


Average 


Avg morbidity 


Avftmof 


in plan 


6-year DFS 


of testing ^ 


■ wttwi 


Total nodal 


0.6386 


0.0 


1500 


MOPP 


0.6610 


0.0 


3000 


"Sandwich" 


0.6819 


0.6 


3500 


Laparotomy 


0.6938 


885 


3*88 


Gallium scan 


0.6942 


869 


3186 


Lymphangiogram 


0.6969 


814 


3249 


Marrow biopsy 


0.6942 


850 


3186 


Liver biopsy 


0.8939 


871 


3t» 



The plan with the highest disease-free-survival is the one begianing with the gallium scan: 
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Negative p = .97 



l + ll p = .50 
s s > Sandwich 



Negative p = .69 



= =>LAP 



6 



>>BMBX 



O 



lll + IV p=.50 



Gallium 



O 



:>MOPP 



l + ll p = .13 



Positive p >: .03 



= >MQPP 



= =T Sandwich 



Negative p = 



Negatifvep*.99> 'k 
= =>LAP V 



Positive p * .31 



= =>LBX 



*b 



III + IV p». 87 



= =>BMBX 



6 



= >MOPP 



Positive p» .01 



»>MOPP 



Positive p = .05 



= =>MOPP 

Altering these factors has led to different diagnostic plans. Once again, further testing before selecting 
treatment gives better average disease-free survival values. However, mere is nbw less emphasis on the tests, 
laparotomy and lymphangiogram, which have been estimated to have increased mortality rates. 



December 1981 



57- 



Hodgkins Decision Analysis 



7. Analysis: Immediate Laparotomy in Young Asymptomatic Patients 

We wished to use decision analysis to answer two related questions: 



1. Are there certain presentations of Hodgkins' disease (HD) which have such a low probability of 
infradiaphragmatie disease that staging laparotomy is not justified? 

2. If negative test results, cannot give sufficient certainty of early stage HD to*fc*tate laparotomy, 
should the patient proceed directly to laparotomy and not be subjected tofother staging tests? 



The decision being evaluated is shown in figure 7,1* which is a^sfflipkrTEST versus TREAT decision tree. 
The test branch for laparotomy, includes its peri-operative and late infective mortality of 1%, and the option 
to treat each stage with its optimum treatment Since me optimum treatment for stage IIIA is being debated, 
we will evaluate the decision separately for the two different treatments which are now being used for IIIA, 
namely total nodal irradiation (TNI) and combined modality therapy (CM). %te«ded«aimVTadiotherapy 
(EM) is evaluated in the TREAT branch for these young asymptomatic patients, as this would be the 
treatment appropriate to their clinical stage. 

If the utility of immediate treatment is higher man that after laparotomy followed by stage-specific treatment, 
the tests are indicated since, if they prove to be negative, the^j^y obvJ^ laparotomy. If, on the other hand, 
laparotomy has the higher utility even when all the tests are negative, then one might proceed direcdy to 
laparotomy, with the knowledge mat testing can never provide enough certainty of early stage disease to allow 
immediate treatment 

This particular analysis concerns asymptomatic patients in the age range 12-39 years, the age group with the 
highest chance of early stage disease. The probabilities of stage used to calculate the utilities are derived from 
the findings listed below • one criterion is taken from each category: 
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Fig. 7.1. 



I + 11 

Pd + li) 



EM 



LAP 



o 



-6 



p(ni) 



IV 



P(IV) 



TNI or CM 



MOPP 



Operative 



mortality (.01) 



t 



l + ll 



Extended 



mantle 



-o 



Pd+ll) 



III 



p(lll) 



IV 



P(IV) 

** When combined modality therapy is being evaluated as the optimal 
treatment for stage IIIA, MOPP salvage of relapse after the single 
modality therapy will be included for fair comparison. 



Histologic subtype - only nodular sclerosis (NS) and mixed cellularity (MC) are considered. 

Male or female 

Involvement of right neck nodes (RN+/RN-) 

Involvement of left neck nodes (LN + /LN-) 

Involvement of mediastinal nodes (MED + /MED-) 

There are 32 possible combinations of these findings; each set of probabilities is further modified in 
accordance with results of all tests prior to laparotomy being negative. For simplicity we will consider only 
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eight of these combinations i.e. for each sex/histology combination we will show the calculated utility for the 
presentations with: 

the most "FAVORABLE" clinical findings - LN-, RN-, MED+ 
the most "UNFAVORABLE" findings - LN+. RN+, MED- 

Table 7.1 shows the average five year disease-free-survival (5 yr DFS) calculated for each branch of the 
decision tree in Figure 7.1, when the treatment for stage HIA after laparotomy feTNL 



Table 7.2 shows the average S yr DFS for each branch of the decision tree when combined modality is used 



Table 7.1. 



PATIENT 



S year Disease Free Survival with Laparotomy followed 
by Single Modality Therapy or with Immediate Extended Maojtye Radiotherapy 



NS FEMALE "FAVORABLE" 
(0.93.0.07,0.0) 

NS FEMALE "UNFAVORABLE" 
(0.85,0.16,0.00) 

NS MALE "TAVORABLE" 
(0.91.0.08,0.00) 

NS MALE "UNFAVORABLE" 
(0.82, 0.17, 0.01) 

MC FEMALE "FAVORABLE" 
(0.88,0.12,0.01) 

MC FEMALE "UNFAVORABLE" 
(0.75,0.22,0.03) 

MC MALE "FAVORABLE" 
(0.82.0.18. 0.01) 

MC MALE "UNFAVORABLE" 
(0.66,0.32,0.02) 



LAP WITH 


1 EXTtMDfETJ 


TNI FOR 


MANTLE 


STAGE r*I 


RADIOTHERAPY 


79. 9% 


78. 9X 


7$.« 


75. 7% 


7*.«% 


78.3% 


77. 9% 


74.0% 


78.9% 


76.5% 



76. 6% 
77.8% 
74.8% 



70.5% 



74.0% 



66.5% 
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for those patients found to be in stage IIIA at laparotomy. For comparison with immediate extended mantle 
radiotherapy, allowance is made for MOPP salvage of patients who relapse after any single modality therapy. 



Table 7.1 



5 year DFS with Laparotomy followed by Single/Combined(IIIA) 
Modality Therapy or with Immediate Extended Mantle Radiotherapy 



PATIENT FINDINGS 
(PROBABILITIES) 



LAP WITH 
CM FOR 
STAGE III 



EXTENDED 

MANTLE 

RADIOTHERAPY 



NS FEMALE "FAVORABLE" 
(0.93.0.07,0.0) 

NS FEMALE "UNFAVORABLE" 
(0.85,0.15,0.00) 

NS MALE "FAVORABLE" 
(0.91,0.08,0.00) 

NS MALE "UNFAVORABLE" 
(0.82, 0.17, 0.01) 

MC FEMALE "FAVORABLE" 
(0.88,0.12,0.01) 

MC FEMALE "UNFAVORABLE" 
(0.75,0.22,0.03) 

MC MALE "FAVORABLE" 
(0.82,0.18, 0.01) 

MC MALE "UNFAVORABLE" 
(0.66,0.32,0.02) 



87 . 7% 



87.1% 



87.5% 



86.8% 



87.2% 



86.1% 



86.8% 



85.5% 



86.7% 



84.1% 



86.1% 



82.8% 



84 . 8% 



79.8% 



82.8% 



76.8% 



For all the combinations of clinical findings considered, and with both single and combined modality 
treatment options, laparotomy is calculated to have the highest average utility. However, for some of the 
more favorable clinical presentations there is only a small difference in outcome between laparotomy and 
immediate treatment. Only when the probability of stage III disease exceeds 0.20, e.g. the 
"UNFAVORABLE" MC presentations, is the difference in average five year DFS between the TEST and 
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i vEAT options above 5%. Other factors than disease-free survival may influence the decision, for example 
.iie opportunity to perform oophoropexy in female patients. 

Tr one cannot obviate laparotomy even when all other staging tests are negative, we feel it is preferable to 
proceed directly to laparotomy once the diagnosis of Hodgkins disease is made in this group of asymptomatic 
patients in the age group between 12 and 39 years. We particularly favor this course of action when certain 
unfavorable clinical findings give an appreciable chance of stage III disease. 
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8. Analysis: Laparotomy in Symptomatic Patients 

In using the Bayesian diagnosis program we found that certain cohorts of symptomatic patients with a positive 
lymphangiogram have a very low probability of early stage disease. This study was carried out to determine 
whether, amongst these cohorts, there were any for whom the risks of staging laparotomy (LAP) outweighed 
the benefits it conferred through knowing the stage accurately. This work was first performed and published 
using the 900 patient database (Rutherford et al, 1980). The present study uses the prior probabilities 
calculated from the 1200 patient database. 

For this study the probabilities of stage were calculated for cohorts of symptomatic patients, with a cohort 
being defined by one criterion from each of the following categories: 

(1) Nodular sclerosis (NS) or mixed cellularity (MC) histologic subtypes 

(2) Age 12-39 years, or older than 39 years 

(3) Male or female 

(4) Involvement/non-involvement left cervical lymph nodes. 

(5) Involvement/non-involvement right cervical lymph nodes. 

(6) Involvement/non-involvement mediastinal lymph nodes. 

All patients were assumed to have a positive lymphangiogram and a negative percutaneous bone marrow 
biopsy. 

There were thus 56 different combinations of findings, for which probabilities of stage were calculated, again 
using a simplified three stage version of the Ann Arbor criteria (Carbone et al, 1971), in which stages IB and 
IIB are combined. There were too few patients with the lymphocyte predominant and lymphocyte depleted 
subtypes for meaningful data in these categories. 
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8.1 Calculation of Decision Making Thresholds 

A decision-making threshold was calculated at which the result of treatment after laparotomy, taking into 
account the mortality of the operation, was equivalent to immediate combination chemotherapy treatment 
with nitrogen mustard, vincristine (Qncovin), procarbazine and prednisone (M0PP) (DeVita et al, 1980). 
Treatments planned after laparotomy were total nodanrT^iatton (Tm) for stages IB AND IIB, and MOW* 
chemotherapy for fllB and IVR. Estimates of me profeabfflty If WW OM^'llte survival (DFS) for a 
given stage with these treatments are shown in Table 8.1. These resai* ai^ otrtamed from data from the 
literature (Aiscnberg et al, 1976, DeVita et al 1980, Goodman et al 1977, Rosenberg et al, 1975). 



Table 8.1. Pensatage S Year Dtsease-Frte SarriMlfet Ti iefi*i . 

TNI MOW 

Stage IB and I IB 70 BO* 

Stage I I IB 26 60 

Stage IVB 0* 36 

* * estimate 

The probability of 5 year disease-free survival after laparotomy is calculated by adding the products of the 

probability of each stage, derived from the computer and the results of die best treatment for that stage; this is 

then corrected for the mortality of the operation (peri-opcrative and post-infostiveXie. 

[ DFS TM.I+II P(I + ") + DFSmopp.III P< IU > + DFS MQPP,IV P( IV )1 (* " mortality) 
If we substitute the appropriate DFS values from Table 8.1 this expression becomes: 

[0.70 p(I+II) + 0.50 p(III) + 0.35 p{IV)] (1 - mortality) 
The probability of 5 year disease- free survival if immediate MOPP therapy is given is: 

DFS MOPP,I+II P( I+II > + DFS M0PP,III P< m ) + DFS MOPP.IV P( IV > 
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Again, substituting the values from Table 8.1 this becomes: 

0.60 p(I+II) + 0.50 p(III) + 0.35 p(IV) 
P(IV) can be expressed as [1 - p(I + H) - p(M)]. The threshold between laparotomy and immediate MOPP 
therapy - found when the post-laparotomy disease-free survival is equal to that after immediate MOPP - can 
then be written as a linear equation. If, for example, one assumes a 1% total mortality from laparotomy, 
substitution of this in the laparotomy expression gives a linear equation for the threshold in terms of the 
probability of stage (I + II) and III: 

9.65 p(I+II) - 0.15 p(IH) = 0.35 
This threshold is displayed in a graph with the probability of stage (I + 11) and stage III on abscissa amd 
ordinate respectively and a line drawn between 1.0 on each axis, to outline the area relevant to the study 
(Figure 8.1). 

Any patient cohort can be represented by a point on this graph in terms of the probability of stage I +11 and 
stage HI. The nearer the origin of the triangle, the higher the probability of stage IV. 

The threshold between laparotomy and MOPP marks off an area where there is a low probability of stage IB 
and IIB. To the right of this threshold line is the area where laparotomy is the best course of action, despite its 
mortality. 

The equation for the threshold between laparotomy and MOPP is affected by the estimated mortality for 
laparotomy. As the estimated mortality of laparotomy increases, this threshold shifts to the right, diminishing 
the area where laparotomy is the preferred course of action, and increasing the area where immediate MOPP 
should be given (Figure 8.2). 

The equation for the threshold if the laparotomy moitality is 2% is: 
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Fig. 8.1. 

Decision thresholds between LAP and MOPP (Threshold i) and LAP and TNI 
(Threshold ii) when LAP mortality is estimated as 1%. 




P0*ll> 



0.3 p(I+II) - 0.3 p(III) ■ 0.7 
and if laparotomy mortality is 3%* toe threshold equation beoomet: 

8.95 p(I+II) - 0.45 p(III) * 1.06 

If there is uncertainty in the results of therapy, the threshold equation reflects this, and the threshold line 
becomes blurred. For example, if our estimates for DFS are off by 0.03 in opposite directions, and the 
difference between the results of the two treatments is at a maximum, the threshold line between MOPP and 
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Fig. 8.2. The effect of increasing LAP mortality estimates oa decision tkwsboHs 



P(lll) 



Threshold (i) 




Threshold (H) 



P0 + II) 



Values for varying LAP mortality are indicated. 



laparotomy is shifted to the left Conversely, when there is the least difference between results for each 
treatment, die threshold line is shifted to the right, making the vatae of laparotomy less. This "blurring" of 
the line is accentuated by increasing mortality of laparotomy (Figure 83). 

If laparotomy has a false negative rate the decision threshold is shifted in such a way as to diminish its value. 
False negative rates of laparotomy of 1% and 5% are considered. In this context "false negative" refers to true 
stage III patients who are incorrectly called stage (I + II), and to stage IV patients mis-staged, with one quarter 
of the latter being called stage (I +11), and the remainder being called stage III. 
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Fig. 8.3. The effect on deciston tlmshoHs of 3% uncertainty m treatment results 



Threshold (i) 



P(lll) .50 




1 ,:- I 

.10 .20 .30 .40 



I ■-• I 3» I t. I OS. I Pr I 
50 .60 .70 .80 .90 1.0 



P(l + ll) 



The dotted lines on either side of tire solid lines bound the areas 
included when 3% uncertainty in treatment results is taken into account. 
(LAP mortality is estimated as 1%. ) 



Analogous threshold lines can be drawn between laparotomy and immediate TNI; these mark off a triangular 
area where there is a very high probability of Stages IB and HB. We found, however, that Aerc were no 
cohorts of symptomatic patients - not even those with the most favorable combinations of findings and 
negative lymhangiogram - whose probabilities fell within the area to the right of the laparotomy/TNI 
threshold, where immediate TNI would be the better option. This was so even when laparotomy mortality 
was increased to 3%, when uncertainty in therapy results was considered and when false negative rates were 
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considered for laparotomy. 

8.2 Results 

A point representing each of the fifty-six sets of probabilities of stage derived by the computer for 
symptomatic lymphangiogram-positive "patients" was plotted on a graph/ like those shown in figures 8.1-8.3 
to determine which decision was appropriate for each cohort These results are presented in tabular form. 
Table 8.2 shows results when the only variable affecting me threshold was the mortality of the staging 
laparotomy. With laparotomy mortality estimated at 1%, six of the 56 cohorts had such a high probability of 
late stage disease that the point representing their probabilities was plotted in the area where MOPP therapy 
could be given immediately. If the mortality of LAP was 2%, and the threshold line moved to the right 
(Figure 8.2), the "immediate MOPP area" included all male MC patients with positive lymphangiograffl, 
together with several other cohorts which represented the less favorable findings for the other categories. If 
the laparotomy mortality was estimated to be 3%, all male patients with positive lymphangiogram are plotted 
in the "immediate MOPP area", together with more than half the cohorts of female patients considered. 

Tables 8.2-8.5 show that the numbe of cohorts of patients for whom immediate MOPP was justified increased 
considerably, when some of the variables influencing the decision, namely the false negative rate for 
laparotomy and the uncertainty of the treatment results, were considered. The uncertainty in treatment 
results influenced the decision most, but considering false negative and fake positive rates for laparotomy also 
increased the number of cohorts for whom immediate MOPP was indicated. 
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Table 8.2. 
SYMPTOMATIC ADULT* PATIENTS WITH POSITIVE LYMPHANGIOGRAM FOR IMMEDIATE MOPP 

LAP. HIST. SEX PATTERN OF NODAL INVOLVEMENT 
MORT TYPE N-N- N+N- N+N+ N+N- N+N+ 

M+ M+ M+ M- M- 



1% 


MC 


M 


- 


>39 


>39 


>39 


>39 


2% 


NS 


M 


>39 


>39 


>39 


ALL 


ALL 




NS 


F 


- 


- 


- 


>39 


>39 




Hv 


M 


ALL 


ALL 


ALL 


ALL 


ALL 




MC 


F 


- 


- 


>39 


>39 


>39 


3% 


NS 


M 


ALL 


ALL 


ALL 


ALL 


ALL 




NS 


F 


- 


>39 


>39 


>39 


ALL 




MC 


M 


ALL 


ALL 


ALL 


ALL 


ALL 




MC 


F 


>39 


>39 


>39 


ALL 


ALL 



*Adult - older than 11 years 

>39 - refers to patients older than 39 years 

M = Mediastinum N ■■ Cervical lymph nodes 

+ = involved with Hodgkins disease 

- = not involved with Hodgkins disease 

i.e. N+N+ = Bilateral cervical node involvement 
N-N- ■ Cervical nodes not Involved 
N-N+ = Unilateral cervical node Involvement - right- or left-sided. 

8.3 Discussion 

From basic clinical information and the results of the lymphangiogram we have been able to predict the 
likelihood of the various pathologic stages in symptomatic HD, and to compare the value of immediate 
chemotherapy with that of laparotomy to a group of patients who have a tow probability of early stage disease. 
With the simple decision-making techniques used here it is possible to analyze the effect of three important 
variables on the decision to perform laparotomy - its mortality for a given patient, the validity of the results of 
laparotomy and the reliability of treatment results on which management decisions depend. Even with the 
conservative figures we have used for these variables, this approach could save an appreciable number of 
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Table 8.3. 

SYMPTOMATIC ADULT* PATIENTS WITH POSITIVE LYMPHANG IOORAM FOR IMMEDIATE MOPP 
IF THERE IS 3% UNCERTAINTY IN TREATMENT RESULTS 

PATTERN OF NODAL INVOLVEMENT 
N-N- N+N- N+N+ N+N- N+N+ 
M+ M+ M^ M- M- 

>39 

ALL ALL 



2% ALL 

3% ALL 



LAP. 


HIST. 


SI 


MORT 


TYPE 




1% 


NS 


M 




NS 


F 




MC 


M 




MC 


F 



>39 


ALL 


ALL 


- 


>39 


>39 


ALL 


ALL 


ALL 


>39 


>39 


>39 



For key to table, see Table 8.2 



Table 8.4. 

SYMPTOMATIC ADULT* PATIENTS WITH POSITIVE LYMP HANGIOGKAM 
FOR WHOM IMMEDIATE MOPP IS, THE 8ITIE8 OPTION 
WHEN LAPAROTOMY IS CONSIDERED TO HAVE A 1% FALSE NEGATIVE RATE 



LAP. 


HIST. 


SEX 


PATTERN OF 


NODAL 


INVOLVEMENT 


MORT 


TYPE 




N-N- N+N- 


N+N+ 


N+N- 


N+N+ 








M+ M+ 


M+ 


M- 


M- 


1% 


NS 


M 


- 


- 


- 


>39 




MC 


M 


>39 


>39 


>39 


>39 


2% 


NS 


M 


>39 >39 


ALL 


ALL 


ALL 




NS 


F 


- 


>39 


>39 


>39 




MC 


M 


ALL ALL 


ALL 


ALL 


ALL 




PH« 


F 


>39 


>39 


>39 


>39 


3% 


NS 


M 


ALL ALL 


ALL 


ALL 


ALL 




NS 


F 


>39 >39 


>39 


ALL 


ALL 




MC 


M 


ALL ALL 


ALL 


ALL 


ALL 




MC 


F 


>39 >39 


ALL 


ALL 


ALL 



For key to table, see Table 8.2 



patients the rigors, risks and expense of a LAP, and of the other tests frequently performed in routine staging. 

In certain patients the risk of laparotomy mortality may well be higher than 1%. Immediate postoperative 
mortality for 2345 patients from 27 different series was 0.5% (Table 1.3); none of these scries reported 
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Table 8.5. 

SYMPTOMATIC ADtiLT^PATIENTS WITH fOSftMVE LYMRHANGIOGRAM 

FOR WHOM IMMEDIATE MOPP IS TaEOBETTER OPTION 

WHEN LAPAROTOMY IS CONSIDERED TO HAVE A 5% FALSE NEGATIVE RATE 



LAP. HIST. SEX PATTERN OF NODAL INVOLVEMENT 
MORT TYPE N-N- N+N- N+N+ N+N- N+N+ 

M*% Mv Mr, M™ M— 

IX NS M - >39 >39 >39 ALL 
MC M >39 >39 ALL ALL ALL 
MC F - - . ■ -. . >39 

__ _______ — ___________________^_^______________. 

2% NS M ALL ALL ALL ALL ALL 

NS F - >39 >39^ >39 ALL 

MC M ALL ALL ALL ALL ALL 

MC F >39 >39 ALL ALL ALL 

3X NS M ALL ALL ALL ALL ALL 

NS F >39 >39 ALL ALL ALL. 

MC M ALL ALL ALL ALL ALL 

MC F >39 ALL ALL ALL ALL 

____•-_-_-_-_■._._-_• — _»«■-*%!»<-» — — «■».»» — «#*_«■_ — ■.«_._»'— -_4-^;_»^-i_. wm T m"^*it* m V »4 4 ft w . W*<^WW ■*-t*,i»w»-' ■»_»,_->-! 

For key to table see legend for Table 8^2" • -■ 



age-specific rates, but there pay be wide^iriatio^ v^th mucJi Jftwer rases in- younger patients and higher rates 
in older patients, or those with other complicating factors. In addition to posHqwrative mortality, there may 
also be a delayed mortalRy due to the fulminant sepsis known to occur in a small proportion of 
splenectomized patients; this usually occurs within 2 years of splenectomy (Krivit, 19f7). When Desser and 
Ultmann (1973) collated results from 1190 patients splenectomkted as part of staging laparotomy, they found 
that late death from overwhelming sepsis, unrelated to treatment-induced ieucopenia or other precipitating 
cause, occurred in 0.5%. Gfiter series report higher rates of post-splencctomy sepsis; for example Singer 
(1973), reviewing 297S splenectomized patients, found the overall k^e death rate from infection was 2.52%; 
even patients who had splenectomy incidental to another operation had an appreciable mortality from this 
complication (0.36%). Furthermore, susceptibility to infection alter splenectomy may be exacerbated by 
treament-induced marrow suppression, (Schimpff & O'ConnelL 1977), and by the defects in cellular 
immunity frequently noted in patients with HD (Desforges et al, 1979). 
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In this study, the mortality of laparotomy is considered to be its only disadvantage, and one must remember 
its other liabilities. In addition to mortality, 6.7% of the 2345 patients in Table 1.3. had serious complications 
of the staging laparotomy; these included such major post-operative problems as subphrenic abscess, 
pulmonary embolism or intestinal obstruction requiring reoperation. 

Uncertainty about results of treatment should influence therapeutic decisions. For example there is no firm 
evidence that any treatment has a particular advantage for stages IB and HB HD; recent trials (Goodman et al 
1977, Rosenberg et al 1978) have not demonstrated a statistical significant difference between TNI and 
combined modality treatment AH reported series of pathologically-staged HB patients contain less than 25 
patients in any particular treatment group (Aisenberg & Qazi, 1976, Fazekas et al 1975, Fuller & 
Madoc- Jones, 1977, Goodman et al 1977, Rosenberg et al 1978, Stoflel *Cox, 1977). In these small groups 
results for TNI vary between 79% and 48% 5 year disease free survival. For a series of 25 patients, 95% 
confidence limits for a disease free survival of 79% range between 90% and 60%, and when disease free 
survival is 48% they range between 72% and 27% (Greenwood & Hartley, 1962). The uncertainty rate we have 
used (3%) in these calculations is, therefore, a very conservative estimate for uncertainty in TNI results. Even 
this modest uncertainty, however, made a considerable difference to the numbers of patient cohorts for whom 
immediate treatment was appropriate. The MOPP chemotherapy regime for treating disseminated disease 
has large series (Aisenberg & Qazi, 1976, DeVita et al, 1980) reported with long-term follow-up and much 
narrower confidence limits; there is, however, little information on its use in localized disease. Some data on 
the use of MOPP for early stages has come from Uganda (Olweny et al, 1978), but this patient group is 
scarcely comparable, since their patients were predominantly young boys, who had minimal clinical staging; 
moreover, separate data for A and B patients is not given. Our figures for MOPP for stage IB and HB are 
estimates, therefore, and subject to uncertainty, as are die estimates for TNI for disseminated disease. 

A further complicating factor in the decision is the accuracy of laparotomy. Although physicians use the 
results of this operation as the final answer, there is likely to be a false negative rate, due to sampling error. 
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This sampling error is difficult to qptntitate; Caiman et al (1977) showed |hat 18 of 88 Hodgkins disease 
patients still had lymphographically-abnormai nodes after LAP. A 1% false negative rate seems realistic for a 
surgeon experienced in this type of surgery; a S% false negative rate may be appropriate for a surgeon not 
experienced in staging laparotomy. 

In spite of these uncertainties in risks and outcome, decision* regarding staging and subsequent management 
must be made. Tables 8*2-8.5 give guidelines for patients who jnay proceed directly to treatment with MOPP, 
with provision to include the possible variables. The same principles of threshold analysis can be applied 
even if different treatments are used. Probabilities of pathological stage for a given patient can be obtained 
from Appendix 1; Bayes Rule can be used to incorporate the results of arty test, using false positive and raise 
negative rates appropriate to the institution in which they have been performed, 'Nejw decision thresholds can 
be derived by substituting the new treatment results in the threshoMequatioiB. 
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9. Problems in Applying Decision Analysis to Hodgkins Disease 

There are some limitations in the Hodgkins disease decision analysis program which may restrict its 
usefulness. 

Utilities have posed the major problem. When we wish to evaluate our plans in terms of five year disease-free 
survival (5 yr DFS) we have had difficulty in finding large series of patients for whom disease-free survival 
data is available. All too often the series are small, with relatively few patients followed through to the five 
year stage, so that the data has considerable uncertainty. For some stages and treatments there is no 
disease-free survival information available, for example for chemotherapy in early stage disease or for 
radiotherapy in advanced disease. These values, therefore, have had to be estimated. 

A further problem with the utilities of plans is that there is no reasonable way to "trade-off' or compare 
morbidity data with disease-free survival data. In the program these are, therefore, simply quantitated and 
expressed separately. Even comparing the different types of mortality is a problem. In our plans the "cost" of 
a post-operative death from laparotomy is balanced against survival disease-free at five years, where the latter 
is used to mean cure. It seems certain, however, that most patients would not regard immediate death and 
relapse at S years as having equal "cost". 

The program is also not useful in those cases with unusual presentations. Since it is dependent on data to 
calculate both the prior and the conditional probabilities of stage, unusual presentations are poorly 
represented and we have little confidence in probabilities calculated from these data because of sampling 
error. The management of such presentations as bilateral axillary node involvement alone, or of inguinal or 
other isolated infra-diaphragmatic involvement is not clear cut For this reason these are just the type of 
patient for whom the physician would like some guidance about the likely spread of disease, yet because of 
the relative rarity of such cases our program does not have sufficient data to give reliable probabilities of 
stage. The same problem arises with patients with the less common histological subtypes - lymphocyte 

December 1981 -75- Haigkins Decision Analysis 



depleted and lymphocyte predominant 

There is also a problem with histologic interpretation. Out prior probabilities of stage are quite strongly 
influenced by the histological subtype of the HD. There is, however, considerable variation amongst 
pathologists in their reporting of the HD histology within the Rye histologic classification (Jones et al, 1977). 
Since our testing of the database showed that the program made accurate predictions of stage, we have 
concluded that this uncertainty in assigning histologic subtypes is not of major importance to our program. 
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10. The Computer Implementation 

The goal of this chapter is to reveal enough of the workings of the Hodgkins disease program to allow an 
interested programmer to reproduce it The following toptes will be addressed: 

1. representations of knowledge about Hodgkins disease, including stages, clinical findings, tests, 
and treatments, and probabilities 

2. data structures for maintaining a description of a patient 

3. data structures for representing decision trees 

4. calculating probabilities of stage 

5. growing decision trees and making decisions 

6. subsidiary programs for displaying and modifying program data 

7. interactive features to deal with the twin problems of low band-width data transmission and 
use of typewriter input by non-expert typists 

8. additional features: threshold analysis, salvage analysts 

10.1 Introduction 

The Hodgkins disease computer system consists of two independent subsystems: a computer program for 
performing decision analysis in the management of patients with Hodgkins, and a database system for 
managing information about 2000 individual Hodgkins disease patients. Both subsystems are large, 
interactive Lisp programs devoted to ease and rapidity of use. Hie operation of these programs from the 
user's point of view has been discussed in other sections. This section deals with the details of the 
implementation of the decision analysis subsystem. 
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10.2 Maclisp as an Implementation Language 

Several features of LISP, and of the Maclisp dialect uv particular, facitoaleddie devetopmentof the Hodgkins 
disease program. LISP provides data structures .appropriate W cepresenting complex structures such as 
decision trees and diagnostic plans, namely s-expressions. Maclisp, together with the text editor Emacs, 
provides a remarkable interactive programming environment, with ex£*tie^<debuggiBg fac&ties; nEhe LISP 
language, augmented by a macro package for iter^n, providcaco^folsinKmires^pispriate to creating and 
manipulating complex, repetitive, and recursive structures: The flexible mput and ^u^nit capabilities of 
Maclisp allow the easy development of dialog between computer and user. 

10 3 Data Structures 

The Hodgkins disease decision analysis program operates: ©n three type* oil Jnlbnnation: knowledge about 
Hodgkins disease, descriptions of the patient under coostderatku^a^declstostreesv 

10 J.l Knowledge about Hodgkins Disease 

The program has rued data structures repwsentwg its- Ho^ The program's 
knowledge of Hodgkins disease cooaists of: 

1. identifiers for important aspects of me disease, including symptoms, tests, and treatments (eg. 
an important aspect of a particular patient's itedgkiaa disease it; its ^ymptematclogy" ~ 
whether the patient is asymptomatic (A) or symptomatic (B)); 

2. qualitative relationships between these aspects of the disease (eg. tile program contains 
information denoting the fact that symptomatology fc an «*»»tl«* iJihital finding in 
Hodgkins disease); and 

3. quantitative (probabilistic) relationships between aspects of the disease (eg. the probability of 
positive left nock nodes in a symptomatic stage HI patient is 0.75). 

This knowledge is stored partly as the values of free (or global) variables in the Lisp environment, and partly 

as property lists. Free variables are used mainly to give names to useful lists of items, such as the list of 
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clinical findings or the list of tests. Examples of such free variables, and their default values, are shown below. 
The names of the variables are on the left, their values, in conventional LISP list notation, are on the right 

variable Value 

*prior-f indings* ' (histologic-subtype symptomatology sex) 

•conditional-clinical-findings* '(age left-neck right-neck mediastinum 

spleen-size livers-scan) 
•clinical-findings* (append *prior-f indings* 

*conditional-cl in ical -findings*) 
•tests* '(gal lag bmbx lbx) 

•all-findings* (append *clinical-f indings* *tests*) 

•treatments* ' (total -nodal mepp extended-mantle 

combined-modal ity) 
•stages* '(i+11 iii iv) 

These lists may be modified under user command. For example, if the user wants to have the program 
consider a treatment different from the four included in 'treatments*, he invokes a request which runs a 
program which adds the name of the new treatment to the list * treatments*. 

Property lists are used to maintain detailed information about each of the findings, tests, and so on. For a 
clinical finding, this information includes the list of values of the finding (e.g. SEX may be one of (MALE 
FEMALE )), plus directions to the program about how to ask about the sex of the patient and how to display it 
to the user. Such information is stored in a disk file, from which it is loaded when a new version of the 
Hodgkins program is constructed. For example, the information about SEX appears in a text file as the 
s-expression below: 

(•load sex 

(kind clinical-specific) 

(results (male female)) 

(inquire (|What is the sex of the patient?|))) 

This expression is evaluated, to give SEX the property list shown below. (The "INQUIRE" andy"KEYLST" 
properties provide the program with information for asking the user questions about the sex of the given 
patient) 
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(RESULTS (MALE FEMALE) 
KIND CLINICAL-SPECIFIC 

INQUIRE (|What is the sex of the patient?|) 
KEYLST ((MALE MIL KtYSTRISG fM»Ve[«tTWr J NALE) 

(FEMALE NIL KEYSTRINS | Female) RETURN 'FEMALE) 
(UNKNOWN NIL KEYSTRING |Unknown| RETURN 'UNKNOWN) 
(|*WHAT IF . .« ?J NIL KEYSTRING |»What if ... ?J 

RETURN T'WHAT-IF) 
( | 'FINDINGS SO FAR| NIL KEYSTRING |«Hnd1ng».*0<far| 

RETURN H^NOJNaS-SQrFAR) 
(|*DISPLAY DATA ^ WU KEYSTRING |*D1$pUy data| 
RETURN DISPUY-DATA))) 

For a test, the detailed information consists of me raw/e of the test, the cost of the test in terms of mortality, 

morbidity, and money, and fa&cotiditiotud probabfHUis relating test results and stage. For a treatment, it 

includes numerical values for survival values, morbidity; and dollar cost 



10.4 Patient Description 

Throughout a decision analysis session, the program maintains a description of the patient, which is 
represented as me property list of the atom PATt ENT -MODEL. It has entries for the known values of clinical 
findings (e.g. SEX=MALE), for the known results of tests previously carried! out, and for the current estimates 
of stage probabilities. Once the decision analysis has been carried out, the set of recommended diagnostic 
plans is added to the patient description. For example, the patient described in the first paragraph of the last 
chapter would be represented internally as the property Ust below: 



(PROBABILITIES (0.486 0.485 0.029) 

A-OR-B B 

HISTOLOGIC-SUBTYPE NS 

SEX MALE 

AGE 11<AGE<40 

LEFT-NECK POSlf f^E-tEFT-NECK 

RIGHT-NECK NEGATIVE-RIGHT-NECK 

MEDIASTINUM NEGATfVE^tEDrftSY INUrf 

SPLEEN-SIZE NORMAL-SIZE-NO-DEFECTS 

LIVER-SCAN N0RMAfc-LIVER-S^M^ 



December 1981 -80- Hottgkim Decision Analysis 



10.5 Probability Calculations 

The program uses Bayes Rule to calculate stage probabilities from the patient description. For reasons 
discussed elsewhere in this article, the three findings SEX, HISTOtOGIC-S^BTYPE, and SYMPTOMATOLOGY 
were selected as the basis on which to estimate the prior probabilities. Once the program has asked for the 
values of each of these findings, it looks up the initial probabilities in a table indexed by the findings. 

The program then asks for the other clinical findings and, using Bayes Rule, updates me probability estimates 
after each answer. Assumptions of conditional independence were made where intuitively reasonable and not 
contradicted by statistical data. Dependence was mcbrporatedin two wsft by clumping dependent findings 
and by explicitly conditioning certain conditional probabilities on the values of other findings. Clumping was 
used in creating the group of findings for initial probabilities. Explicit conditioning was used to deal with the 
dependence of several findings on the symptomatology of the patient Further, several of the findings were 
thought to be tests of more fundamental findings such as fiver or spleen mvolvenient Tn these cases, a two 
stage version of Bayes Rule was used, as described in Section 2.1. 

Our implementation of Bayes Rule in Maclisp calculates the list of posterior probabilities from the 
conditionals and the list of priors, following the mathematical statement of Bayes Rule, shown below for the 
probability of stage I +11 given positive right neck nodes (+RH);. F , 

PM+TTi+mm - pf+RNii+riWrf+r/ri ' 

Hli+iii+KHj - p( +R |i|I+II)*P(I+II) + P( +6M 1 1^1 1 ) ff ( ^ 1 1 1 + ^ P(+RN|IV)*P(I+IV) 
This calculation is carried out by the function Bayes, whose definition in Lisp appears below: 



(defun Bayes (finding prior) 

(unitize ; normalize the list 

(for prob in prior as stage in '(I+II III IV) 
collect 
(times (Conditional-Probability-of finding stage) prob))))) 

;of the products of priors and conditionals 

For example, if the prior probabilities are '(.3 .3 .4) corresponding to stages ...' (I+J I III IV) in a B 
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patient with a certain set of clinical findings, then the list of posterior probabilities given that the patient has a 
positive right neck node is (b ayes '+RN '(.3 .3 .4)). 

The calculation of this quantity proceeds as follows: 

1. finding is bound to '+RN 

2. prior is bound to '(.3 .3 .4) 

3. For each p rob in prior corresponding to the stages, the following product is calculated: 

P( +RN| stage ) *prob 

The Bst of these products is constructed, and men normalized to have sum 1.0 by the function 
unitize. This is exactly equivalent to dividing by the usual denominator in the expression 
for Bayes Rule, since that denominator is just me sum of the uanonnalized products. 

10.6 Decision Trees and DUgaestic Phw 

Once all of the known information about the patient has been used to calculate me stage probabilities, me 
program constructs diagnostic plans for the patient from the available tests and treatments. A diagnostic plan 
is represented by a recursive bst structure according to the following BNF-like syntax: 

Plan := (Utillty-Swmary Utility -Data Branches) 
Utility-Stnaa&ry :» (Utility . Action) 
Utility, is (he numeric expected utility of me plan 
Utility-Data is a breakdown of utility by attributes 1 
Action :» Treatawnt | Test 
Treatment := TNI | HOPP | EM 



1. Though space in the structure has been kit for this item, the current version of the program docs not use multiple utility attributes 
except when summarizing an entire plan at top-levcL The space is filled with ttie atom 'utiBty-data-ptace-hokier. 
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Test := BMBX | LAG | LBX | GAL | LAP 

Branches := (Branch) | (Branch . Branches) 

Branch := (Test-Result Probability-of-Result Posteriors Plan) 

Test-Result is a result of specified test Eg. if test is BMBX, then Test-Result 
may be either +BMBX or -BMBX. 

Probability-of-Result is a numerical value representing the probability that the 
result will occur, given the prior probabilities at that point in die tree. 

Posteriors is the list of probabilities after the result of the test has been taken into 
account 

Plan (second appearance) is the plan which is optimal if die given test result occurs 

For example, the plan shown below is represented internally as die list structure following it 

Plan 1: The following diagnostic plan has an estimated DFS of 0.5873 for an 
A patient with probabilities: I+II 0.0337 III 0.3716 IV 0.5946 

/ 0.04 I+II EM 

/ 0.89 - LAP | 0.42 III TNI 

| \ 0.64 IV MOPP 

BMBX j 

I 

\ 0.11 + MOPP 



Internal representation: 

((0.5872 . BMBX) 
ut i 1 i ty-data-pl ace-hol dec 2 
((-BMBX 0.8885 

(0.0379 0.4182 0.5438) 
((0.5928 . LAP) 
utility-data-place-holder 
((PATH-STAGE-I+II 0.0380 

(1.0 0.0 0.0) 

((0.82 . EXTENDED-MANTLE))) 
(PATH-STAGE-III 0.4183 

(0.0 1.0 0.0) 
((0.64 . TOTAL-NODAL))) 
(PATH-STAGE-IV 0.6438 

(0.0 0.0 1.0) 
((0,55 .MOPP)))))) 
(+BMBX 0.1115 (0.0 0.0 1.0) ((0.55 .MOPP))))) 

The plans are created by exhaustively tracing a large decision tree of possibilities using a computer 



2. See previous footnote. 
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implementation of "averaging out and folding back". The initiator of tree analysis is the function 
Expand-Top-Level -Choice-Node, which sprouts subtrees beginning with each of the possible 
treatments (listed in "treatments*), the special test LAP, and other tests not already done or otherwise 
contraindicated (t e s t s - 1 e f t), and evaluates the plan optionally suppfeitby tte *•#* 

(defun Expand-Top-Level -Choice-Mode (tests-left prior Tap-donef) 
(append 
-.Treatment plans 
(for treatment 1n "treatments* 

collect 

(neons (cons (Treatmeht-E-u" treatment prior) treatment))) 
;Lap plan 

(neons (Lap-Plan prior)) 
;Test plans 
(for tests on tests -left as test ■ (car tests) 

collect (E*pa$dHph|rt^1<^ te^t^^r^^cTtfr: tests))) 
-, User plan' '-' ■:-••■■-.■* v -,i-- ,s ' : ' r! ' 

(and (get 'controller 'user-plan-fora^ T . . .. 

(neons (Instantiate-Plan plan-fcjrm pMpri):)))) 

Expand-Chance-Node is called by the top-level node expander to grow the subtree beginning with a 
particular test It constructs branches for each of the possible results of the tests, expands each branch, and 
calculates the expected utility for the subtree. It returns to die calling program the expected utility and the 
structure of the resulting subtree. 



(defun Expand-Chance-Node (test prior tests-left) 
(for res in (test-results test) 

bind (cond-prob 0.0) posterior branch (utility 0.8) 
(new-tests (cdr tests-left);) r 

collect __ ■■[*■'"> '■> ' '■ ■' "■•'■-, 

(setq cond-prpb (Conditjohil^Proiaftl^ity-of res prior) 
posterior (Bayes ' j %i ftrior) 
branch (1f (eq test 'lap) t '^ ,. : 

(Choose-Best-Treatmeiit posterior) 
(Expand-Cho1ce-NOda eesteMor new-tests)) 
utility (plus utility (t1m|s 8 l|hd*p^66 j flliir6Mnch)))) 
(11st res cond-prob posterior brin^hj * 

returning (11st (cons (Aecoiint-For-Hoiftality test utility) test) 
'ut1lhy-dita-0aci^hefder- ' 
result))) 

Expand-Cho 1 ce-Node constructs a subtree for each remaining test, beginning with that test It evaluates 
each subtree, and returns to the calling program the subtree with the highest expected utility. 
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(defun Expand-Choice-Hede (prior tes^s-left) 
(if (null tests-left) 

(Treatment-vs-Lap prior) 

(for tests on tests-left as test * (car tests) 

bind (best-treatment (Treatment-vs-Lap prior)) 
choose-best (Expand-Chance-Node test prior tests-left) 
according-to '(lambda (x y) flessp ^caar x} (caar y))) 
returning (if (lessp (caar result) (caar best-treatment)) 
be st -treatment 
result)))) 

(defun Treatment-vs-Lap (prior) 

(let ((lap-plan (Lap-Plan prior)) 

(best-treatment-plan (Choose-Best-Treatment prior))) 
(if (greaterp futility lap-plan) ( :utility Jbest-treatment-plan)) 
lap-plan 
best-treatment-plan))) 

(defun Choose-Best-Treatment (prior) 
(for treatment in 'treatments* 

choose-best (cons (Treatment-E-U treatment prior) treatment) 
according-to '(lambda (pi p2) (lessp (car pi) (car p2))) 
returning (neons result))) 

For an example of a program trace showing these programs being rati to create a diagnostic plan, see 
Appendix 4. 



10.7 Interactive Features 

We paid particular attention to the interactive features of the program for two reasons. First, our "customer" 
is typically a doctor with little expertise in typing, so we want to minimize required input to the program. 
Secondly, access to the timeshared computer which runs the Hodgkins program is via slow-speed (30 or 120 
char/sec) dial-up lines, hence it is important for output to be efficient as well as informative. 

10.7.1 Input 

To minimize typing without using graphical devices, we use command completion and tabular data entry. The 
former capability is used when asking the user a menu-selection question, a question with a fixed set of 
possible answers. The user typically has to type only one or two characters to uniquely determine a response 
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-- the program supplies the remaining characters. A tenpMfied vetsien of INTStLlSFs ASKUSER package 
was used to implement this feature. 

Tabular data entry is invoked when the user is required to supply or modify a table of numbers. For example, 
if the user wants to change the survival values for Hodgkias disuse treatments, die computer displays the 
values as shown below: 





I+IIA 


I+IIB 


HI A 


me 


IVA 


IVB 


TOTAL -NODAL 


.81 


.7 


.64 


.26 ; 


.1 


.06 


MOPP 


.7 


.8 


.62 


. . ..».- ! :: 


.55 


.35 


EXTENDED-MANTLE 


.82 


.62 


.*"" 


.16 


.0 


.0 


CM 


.87 


.87 


.86 


.66 


.55 


.35 



This feature allows the user to "edit" a table of numbers using a **reaf-^nle",ed|to<'. The user moves a pointer 
or cursor up or down or across die table by means of smgle chaiacter ccflMnands, and then changes the 
number at the pointer by typing in the new value. 

For example, suppose the user wants to change the 5-year DFS figure for MOPP in IV A disease from 0.55 to 
0.57. When the table is presented to the user, the cursor is in the upper left hand corner. Then the user types 
"D" to move the cursor to the second row and "R" 4 times to move die cursor to the fifth column. Then he 
simply types die new number over die old one. The new enoy becomes permanent when the user types any 
character that is not a digit, " + ", ■"-", or ".". 

10.7.2 Output: Display of Diagnostic Plans 

The primary output of the program is the diagnostic plan for the patient In order to achieve the multiple 
goals of ease of comprehension, efficiency of display, and completeness of information, we employ two 
distinct representations of diagnostic plans: branching tree and outline feemat. 

The branching tree format has been presented in figures throughout the paper, e.g. in section 11.6. The plan 
in that section beginning with RMRX is displayed in outline format as follows: 
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Perform BHBX 

1.1 -BMBX with P » 0.89. Posteriors - 0.04 0.42 0.64 . Then: 
Perform LAP 

1.1.1 PATH-STAGE-I+II with P = 0.04. Posteriors * 1.00 0.00 0.00. 
Then: Perform EXTEUDED-MAMTLE 

1.1.2 PATH-STAGE-Iil with P * 0.42. Posteriors = 0.00 1.00 0.00. 
Then: Perform TOTAL-NODAL 

1.1.3 PATH-STAGE-IV with P * 0.54. Posteriors * 0.00 0.00 1.00. 
Then: Perform MOPP 

1.2 +BMBX with P» 0.11. Posteriors ■ 0.00 0.00 1.S0. 

Then: Perform MOPP 

The advantage of this notation is that it can provide more detailed information than the branching tree 
format. 

In its Lisp implementation, the outline display method is somewhat simpler man the branching tree format 
The former utilizes a simple depth-first algorithm to traverse the tree in combination with a simple text 
formatting program which ensures that lines do not exceed the width of the screen. The tetter, because it 
must satisfy more constraints, uses a more complicated algorithm to ensure mat a given tree will fit on a 
screen both horizontally and vertically. 



10.8 Additions to the Basic Hodgkins System 

The basic modules of the Hodgkins system acquire the patient description, calculate probabilities of stage, 
and construct, evaluate and display decision trees. A number of features were added to the basic Hodgkins 
disease to perform other tasks associated with the decision analysis. 

10.8.1 Tree Acquisition 

Occasionally, the user will want to evaluate a diagnostic plan of his own choosing, to see how it stacks up 
against the plans selected by the program. We provide a facility to allow the user to enter the structure of his 
plan into the computer, so that the plan may be evaluated and displayed along with the program's selections. 

The program requires the user to enter the plan depth first, using display formats simiU r to those used for 
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output of program-generated plans. The program begins by asking the user for the first test in the plan. 
Given the test, and its known possible results, die program can display the first level of a diagnostic plan, 
showing the test and branches corresponding each of the results. For each of the result branches in turn, the 
program asks for the action -- test or treatment -- to take next If the action is a treatment, the branch is 
terminated. Otherwise, the action is a test, so the program recursively asks aibmife each <rf its results. Such 
questioning continues until all leaves of the tree are filled in. 

To integrate this facility into the remainder of the decision analysis system, there are program to calculate the 
utility of a prespecified diagnostic plan. Each time plans are constructed and compared, the utility of the 
user-supplied plan is calculated and ranked with the others. 

10.8.2 Sensitivity and Threshold Analysis 

The program contains facilities for performing sensitivity and threshold analysis. Sensitivity analysis is the 
study of the effects of small perturbations of the data on die results of decision analysis. For example, die 
expected utility of doing laparotomy depends on die cost, in terms of mortality, of subjecting the patient to 
the operation. Sensitivity analysis can determine what diagnostic plans would be recommended for mortality 
estimates ranging below and above the current estimate. 

Sensitivity analysis is implemented as an additional top-level program on die Hodgkins disease system. This 
top-level program varies selected parameters and calculates the optimal plan for each value. Parametric 
variation may be of two types: numeric and non-numeric. In numeric variation, numeric parameters such as 
conditional probabilities are varied over a range. For example, the conditional probability of right-neck 
involvement might be caused to range from 0.4 to 0.6 to see what effect this has on the test/treatment 
decision. In non-numeric variation, the value of a rinding is caused to vary over all possible values to see how 
the finding affects a decision. 
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Appendix I - Comprehensive List of Probabilities for Hodgkins Disease Patients 

A large amount of information important to decision-making in Hodgkins disease has been compiled as the 
basis of the decision analysis computer program. In order to make this information available to those who 
may not have access to our program, we present the following tables of probabilities of stage for 

1. PRIOR PROBABILITIES based on the three interdependent findings of sex, histological subtype and the 
presence (or absence) of symptoms. 

2. CONDITIONAL PROBABILITIES for rmdingjstage which are used to modify these prior probabilities. 
Conditional probabilities are given for both the clinical findings, and the test results. 

3. POSTERIOR PROBABILITIES for all combinations of the clinical findings. 

KB. Although probabilities of pathologic stage are given foe all histologic subtypes and for all age groups, die 
numbers of patients in the "less than 12 years!' age group, and, with the ^^phoj^te predominant" and 
"lymphocyte depleted" histologic subtypes are limited, so probabilities involving these parameters have 
considerable uncertainty. It must also be realized that these probabilities apply only to supra-diaphragmatic 
presentations of Hodgkins disease. 



The following abbreviations have been used: 

A = Asymptomatic 

NS = Nodular sclerosis 

LO - Lymphocyte depleted 

M » Male 

+ = Positive 

LN = Left cervical lymph nodes 

MED = Mediastinum 



B = Symptomatic 
MC * Mixed cellularity 
LP = Lymphocyte predominant 
F * Female 
- ■ Negative 
RN * Right cervical lymph nodes 
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1. PRIOR PROBABILITIES - for interdependent findings HI5TGL0GY/SYMPTOMS/SEX 









STAGE 


STAGE 


STAGE 








I+II 


III 


IV 


NS 


A 


FEMALE 


0.72 


0.27 


0.01 


NS 


A 


MALE 


0.66 


0.29 


0.06 


NS 


B 


FEMALE 


0.66 


0.37 


0.08 


NS 


B 


MALE 


0.34 


0.50 


0.18 


MC 


A 


FEMALE 


0.66 


0.36 


0.09 


MC 


A 


MALE 


0.44 


0.47 


0.09 


MC 


B 


FEMALE 


0.60 


0.66 


0.04 


MC 


B 


MALE 


0.20 


0.44 


0.36 


LP 


A 


FEMALE 


0.80 


0.18 


0.02 


LP 


A 


MALE 


0.73 


0.21 


0.06 


LP 


B 


FEMALE 


0.48 


0.41 


0.11 


LP 


B 


MALE 


0.3S 


0.39 


0.26 


LD 


A 


FEMALE 


0.41 


0.47 


0.13 


LD 


A 


MALE 


0.28 


0.43 


0.29 


LD 


B 


FEMALE 


0.36 


0.48 


0.17 


LD 


B 


MALE 


0.23 


0.42 


0.36 
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2. CONDITIONAL PROBABILITIES FOR FINMNG4STA6C 

AGE V4 

Younger than 12 years I+II 0.03 III 0.06 IV 0.08 

12 to 39 years I+U0.84* Iff 8.74 IV, 8*46 

Older than 39 years I+II 0.14 III 0.23 IV 0.47 

LEFT NECK NODAL INVOLVEMENT 

LN+ I+II 0.680 n III 8.76 IV, 8.73 

LN- I+II 0.32 III 0.26 IV 0.27 



RIGHT NECK NODAL INVOLVEMENT s } ^ c ; ,* 

RN+ 1*110.47; 111 8.49 IV 0.70 

RN- I+II 0.63 III 0.61 IV 0.30 



MEDIASTINAL NODAL INVOLVEMENT SJ ; 

MED+ I+II„ 0.62s m 8.63 IV> ftv63 

MED- I+II 0.38 III 0.47 IV 0.47 



SPLEEN SCAN 

Normal I*fl t 0.93u u If* # 0.64 J V =8,49 

III B 0.66 

Enlarged >15 cm I+II 0.07 III A 0.28 IV 0.32 

r.;.. XJJ.f 0.29 

Filling defects L+JI 0.00s f Iff # 0.09 IV 8.*1 

III B 0.10 

Enlarged with filling defects 

I+II 0.00 III A 0.08 IV 0.08 
III B 0.07 



LIVER SCAN 

-ve I+II 0.23 III 0.23 IV 0.66 

+ve I+II 0.77 III 0.77 IV 0.46 



PERCUTANEOUS LIVER BIOPSY 

-ve I+II 1.00 III 1.00 IV 0.80 

+ve I+II 0.00 III 0.86 IV 0.20 
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LIVER BIOPSY VIA PERITONEOSCOPY 

-ve I+II 1.00 III 1.00 IV 0.39 

+ve I+II 0.00 III 0.00 IV 0.61 



BONE MARROW BIOPSY 

-vo I+II 1.00 III 1.00 IV A 0.81 

IV B 0.56 

+ve I+II 0.00 III 0.00 IV A 0.19 

IV B 0.44 



GALLIUM SCAN (Abdominal nodes) 
-ve 
+ve 

LYMPHANGIOGRAM 
+ve 

equivocal 
-ve 



I+II 0.90 


III A 0.65 


IV A 0.61 




III B 0.56 


IV B 0.54 


I+II o.io 


III A 0.35 


IV A 0.39 




III B 0.44 


IV B0. 48 



I+II 0.18 


III A 0.55 


IV A 0.60 




III B 0.57 


IV B 0.70 


I+II 0.23 


III A 0.16 


IV A 0.14 




III B 0.12 


IV B 0.12 


I+It 0.69 


III A 0.20 


IV A . 26 




III B 0.21 


IV B 0.18 
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Probability 


(I+II) 


III 


IV 


ASYMPTOMATIC (A) PATIEHTS 








NS M AGE<12 








LN+RN+MED+ 


0.5 


0.37 


0.13 


LN+RN+MED- 


0.41 


0.43 


0.16 


LN+RN-MED+ 


0.56 


0.38 


0.06 


LN+RMrMED- 


0.47 


0.48 : 


0.07 


LH?RiM^ED+ 


0.58 


0.3 


0.12 


tNrRN+MED- 


0.49 


0.37. 


0.15 


LN-RMED+ 


0.64 


0.31 


0.05 


NS M 11<A6E<40 








LN+RN+MED+ 


0.69 


0.27 


0.04 


LN+RN+MED- 


0.61 


0.34 


0.06 


LM+RH-MED+ 


0.73 


Mf ..■ 


0.01 


IN+RM-MED- 


0.65 


0.3$ 


0.02 


LN-Ril+MED+ 


0.76 


0.21 


0.03 


INtRN+MED- 


0.69 


0.27 


0.04 


LN-RN-MED+ 


0.79 


0.2 


0.01 


NS M 39<AGE 








LN+RN+MED+ 


0.49 


0.35 


0.16 


iN+RN+MED- 


0.4 


0.41 


0.19 


LN+RUrMED+ 


0.56 


0.37 


0.07 


UM-RN-MED- 


0.47 


0.46. 


0.08 


LN~Rjl+MED+ 


0.57 


0.29 


0.15 


INrRN+MED- 


0.47 


0.38 


0.18 


LN-RN-MED+ 


0.64 


0.3 


0.06 


NS F AGE<12 








LN+RN+MED+ 


0.6 


0.36 


0.04 


LN+RN+MED- 


0.51 


0.45 


0.05 


LN+RN-MED+ 


0.63 


0.36 


0.02 


LN+RN-MED- 


0.54 


0.44 


0.02 


LN-RN+MED+ 


0.67 


0.29 


0.03 


LN-RN+MED- 


0.59 


0.37 


0.04 


LN-RN-MED+ 


0.71 


0.28 


0.01 


NS F 11<AGE<40 








LN+RN+MED+ 


0.75 


0.24 


0.01 


LN+RN+MED- 


0.67 


0.31 


0.01 


LN+RN-MED+ 


0.77 


0.23 


0.0 


LN+RN-MED- 


0.7 


0.3 


0.0 


LN-RN+MED+ 


0.81 


0.18 


0.01 


LN-RN+MED- 


0.74 


0.26 


0.01 


LN-RN-MED+ 


0.82 


0.17 


0.0 



December 1981 ■ 94 • Hodtkins Decision Analysis 



NS F 39<AGE 








LN+RM+MED+ 


0.59 


0.36 


0.05 


LN+RN+MED- 


0.5 


0.44 


0.06 


LN+RN-MED+ 


0.63 


0.35 


0.02 


LN+RM-MED- 


0.54 


0.43 


0.02 


LN-RN+MED+ 


0.67 


0.29 


0.04 


LN-RN+MED- 


0.59 


0.36 


0.05 


LN-RN-MED+ 


0.71 


0.28 


0.02 


MC H AGE<12 








LN+RN+MED+ 


0.29 


0.5 


0.21 


LN+RN+MED- 


0.22 


0.55 


0.23 


LM+RN-MED+ 


0.35 


0.56 


0.1 


LN+RN-MED- 


0.27 


0.62 


0.11 


LN-RN+MED+ 


0.36 


0.44 


0.2 


LN-RN+MED- 


0.28 


0.49 


0.23 


LN-RN-MED+ 


0.42 


0.48 


0.09 


MC H 11<AGE<40 








LN+RN+MED+ 


0.48 


0.45 


0.07 


LN+RN+MED- 


0.39 


0.52 


0.08 


LN+RN-MED+ 


0.52 


0.46 


0.03 


LN+RN-MED- 


0.43 


0.53 


0.03 


LN-RN+MED+ 


0.57 


0.37 


0.07 


LN-RN+MED- 


0.47 


0.46 


0.08 


LN-RN-MED+ 


0.61 


0.37 


0.03 


MC M 39<AGE 








LN+RN+MED+ 


0.27 


0.47 


0.26 


LN+RN+MED- 


0.21 


0.52 


0.28 


LN+RN-MED+ 


0.34 


0.54 


0.12 


LN+RN-MED- 


0.26 


0.6 


0.13 


LN-RN+MED+ 


0.34 


0.41 


0.25 


LN-RN+MED- 


0.26 


0.48 


0.27 


LN-RN-MED+ 


0.42 


0.47 


0.11 


MC F AGE<12 








LN+RN+MED+ 


0.37 


0.4 


0.23 


LN+RN+MED- 


0.29 


0.46 


0.26 


LN+RN-MED+ 


0.45 


0.44 


0.11 


LN+RN-MED- 


0.36 


0.52 


0.12 


LN-RM+MED+ 


0.44 


0.34 


0.22 


LN-RM+MED- 


0.36 


0.39 


0.25 


LM-RN-MED+ 


0.53 


0.37 


0.1 


MC F 11<AGE<40 








LN+RM+MED+ 


0.59 


0.33 


0.07 


LN+RH+MEO- 


0.5 


0.41 


0.09 


LN+RN-MED+ 


0.64 


0.33 


0.03 


LN+RN-MED- 


0.55 


0.41 


0.04 


LM-RN+MED+ 


0.67 


0.27 


0.07 


LN-RN+MED- 


0.58 


0.34 


0.08 


LN-RN-MED+ 


0.71 


0.26 


0.03 
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MC F 39<A6E 

LN+RM+MED+ 
LN+RN+MED- 
LN+RM-MED+ 
LN+RN-MEO- 
LN-RM+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

LP M AGE<12 

LN+RM+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RM+MED- 
LM-RN-MED+ 

LP M 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LM+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LH-RM-MED+ 

LP M 39<AGE 

LM+RH+MED+ 
LN+RN+MED- 
LN+RM-MED+ 
LN+RN-MEO- 
LM-RM+MED+ 
LH-RN+MED- 
LN-RM-MED+ 

LP F AGE<12 

LM+RN+MED+ 
LN+RN+MED- 
LM+RM-MEO+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RH+MED- 
LN-RN-MED+ 

LP F 11<AGE<40 

LH+RN+MED+ 
LN+RN-t-MED- 
LN+RN-MED+ 
LN+RH-MED- 
LM-RM+MED+ 
LN-RH+MED- 
LM-RN-MED+ 



0.35 


0.37 


0.28 


0.27 


0.42 


0.31 


0.44 


0.43 


0.13 


0.35 


0.5 


0.16 


0.42 


0.32 


0.26 


0.34 


0.36 


0.3 


0.52 


0.38 


0.12 


0.57 


0.27 


0.16 


0.48 


0.33 


0.19 


0.65 


0.28 


: 0.07 


0.56 


0.36 


0.09 


0.64 


0.22 


0.14 


0.55. 


0.27 


0.18 


0.72 


0.22 


0.08 


0.76 


0.19 t 


0.04 


0.69 


0.26 


0.08 


0.8 


0.19 


0.02 


0.73 


0.26 


0.02 


0.82 


0.15 


0.04 


0.76 


0.2 


0.06 


0.85 


0.14 


0.01 


0.56 


0.26 


0.19 


0.46 


0.31 


0.23 


0.84 


0.28 


0.09 


0.56 


0.34 


0.11 


0.62 


0.21 


0.17 


0.53 


0.26 


0.21 


0.71 


0.22 


0.08 


0.69 


0.26 


0.06 


0.61 


0.32 


0.08 


0.73 


0.24 ' 


0.02 


0.66" 


0.31 


0.03 


0.76 


- 0.19 


0.05 


0.68 


0.25 r 


; 0.07 


0.79 


0.19 f 


0.02 


0.83 


0.16 


0.01 


0.77 


0.21 


: 0.02 


0.85 


0.16 , 


0.01 


0.79 


0.2 


0.01 


0.87 


0.12 


0.01 


0.82 


0.16 


0.02 


0.89 


0.11 


0.0 
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LP F 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MEO+ 

LD M AGE<12 

LN+RM+MED+ 
LN+RM+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RM-MED+ 

LD M 11<AGE<40 

LN+RH+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

LD M 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

LD F AGE<12 

LN+RN+MED+ 
LM+RH+MED- 
LN+RM-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RM-MED+ 

LD F 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RH+MED+ 
LN-RN+MED- 
LN-RN-MED+ 



0.68 


0.24 


0.08 


0.6 


0.31 


0.1 


0.73 


0.24 u 


0.03 


0.66 


0.31*- 


0.04 


0.75 - 


0.19 


0.06 


. 67 


0.24- 


0.08 


0.79 


0.18 


0.03 


0.14 


0.35 


0.51 


0.1 


0.36 


0.54 


0.21 


0.49 ; ; 


0.3 


0.16 


■-■ 0.62 


0.32 


0.18 


0.31 


0.61 


0.13 


0.33 


0.54 


0.27 


0.44 


0.29 


0.33 


0.43 


0.24 


0.25 


0.48 


0.27 


0.4 


0.49 ? 


0.11 


0.32 


0.56 


0.13 


0.4 


0.37 


0.23 


0.31 


0.42 


0.26 


0.48 


0.41 


0.11 


0.12 


0.3 


0.67 


0.09 


0.32 


0.6 


0.2 


0.45 


0.36 


0.15 


0.48 


0.37 


0.16 


0.27 


0.67 


0.11 


0.29 


0.6 


0.25 


0.4 


0.36 


0.25 


0.47 


0.28 


0.19 


0.51 


0.31 


0.32 


0.65 


0.14 


0.24 


0.61 


0.18 


0.31 


0.41 


0.28 


0.24 


0.46 


0.31 


0.39 


0.48 " 


0.13 


0.45 


0.45 


0.1 


0.36 


0.52 


0.12 


0.5 


0.46 


0.04 


0.41 


0.54 


0.06 


0.53 


0.37 


0.1 


0.44 


0.45 


0.11 


0.58 


0.38 


0.04 
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LD F 39<AGE 








LN+RN+MED+ 


0.23; 


0.43 


0.34 


LM+RN+MED- 


0.17 


0.46 , 


0.36 


LN+RN-MED+ 


0.31 


0.62 


8.17 


LN+RN-MCD- 


0.24 


0.58 


0.19 


LN-RN+NEIH- 


0.29 


0.38 £ 


6.33 


LM-RM+MED- 


0.22 


0.42 , 


. 0.36 


LN-RH-MED+ 


0.38 


0.46 i 


0.16 


SYMPTOMATIC (B) PATIEMTS 








MS M AGE<12 








LM+RN+MED+ 


0.19 


0.47 


0.34 


LN+RM+MED- 


0.14 " 


0.6 r.; 


0.36 


LN+RN-MED+ 


0.26 


0.57 *■ 


0.17 


LN+RN-MED- 


0.19 


0.62 /;' 


0.18 


LM-RN+MED+ 


0.25 


0.42 


0.33 


LN-RM+MED- 


0.18 


0.46 


0.36 


LN-RN-MED+ 


0.32 


0.51 


0.17 


NS M 1KAGE<40 








LR+RM+MED+ 


0.38 


0.49 


0.13 


LN+RM+MED- 


0.3 


0.65 , 


0.16 


LM+RM-MEO+ 


0.43 


0.61 


0.06 


LM+RM-MEO- 


0.34 


0.59 


0.07 


LN-RN+MEO+ 


0.48 


0.42 ,-> 


0.13 


LN-RN+NED- 


0.37 


0.48 


0.16 


LN-RN-MED+ 


0.62 


0.43 


0.06 


NS M 39<AGE 




,: 




LN+RN+MED+ 


0.18 


0.43 


0.39 


LN+RN+MED- 


0.13 


0.45 .. 


0.42 


LN+RM-MED+ 


0.26 


0.56 


n 0.21 


LM+RN-MED- 


0.19 


0.59 


-s 0.22 


LN-RM+MED+ 


0.23 


0.38 - 


0.39 


LM-RN+MED- 


0.17 


0.41 


0.42 


LN-RN-MED+ 


0.31 


0.49 


0.2 


NS F AGE<12 




v'-. 




LN+RN+MED+ 


0.39 


0.42 


0.19 


LN+RN+MED- 


4 0.3 


r» 0.48 J 


0.22 


LN+RN-MED+ 


0.48 , 


/■vO.48 :u 


0.09 


LN+RN-MED- 


0.37 


0.63 & s 


n 0.1 


LN-RM+MED+ 


0.46 , 


a .0.36 ■-,< 


0.18 


LN-RN+MED- 


0.37 


0.42 


0.21 


LN-RN-MED+ 


0.54 


0.38 


0.08 
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NS F 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

NS F 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

MC M AGE<12 

LN+RN+MEO+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RH+MED- 
LN-RN-MED+ 

MC M 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RM-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MEO- 
LN-RN-MED+ 

MC M 39<A6E 

LN+RM+MED+ 
LN+R«+MEO- 
LN+RN-MED+ 
LN+RM-MED- 
LN-RM+MED+ 
Hi-RH+MEO- 
LN-RM-MED+ 

MC F AGE<12 

LH+RN+MEO+ 
LN+RN+MED- 
LN+RM-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 



0.6 


0.34 


0.06 


0.51 


0.42 


0.07 


0.64 


0.34 


0.02 


0.55 


0.42 


0.03 


0.67 


0.27 


0.05 


0.59 


0.35 


0.07 


0.71 


0.27 


0.02 


0.37 


0.4 


0.23 


0.29 


0.45 


0.26 


0.45 


0.44 


0.11 


0.36 


0.62 


0.12 


0.45 


0.34 


0.22 


0.36 


0.39 


0.26 


0.53 


0.37 


0.1 


0.09 


0.33 


0.58 


0.07 


0.34 


0.6 


0.15 


0.49 


0.36 


0.11 


0.52 


0.37 


0.12 


0.3 


0.58 


0.08 


8.31 


0.61 


0.19 


0.45 


0.36 


0.24 


0.45 


0.3 


0.18 


0.49 


0.33 


0.31 


0.54 


0.16 


0.24 


0.8 


0.16 


0.3 


0.4 


0.3 


0.23 


0.44 


0.33 


0.38 


0.47 


0.14 


0.08 


0.28 


0.64 


0.06 


0.29 


0.66 


0.14 


0.45 


0.41 


0.1 


0.47 


0.43 


0.1 


0.28 


0.64 


0.07 


0.27 


0.66 


0.18 


0.41 


0.42 


0.36 


0.64 


0.1 


0.28 


0.61 


0.11 


0.4 


0.66 


0.04 


0.32 


0,64 


0.05 


0.44 


0.47 


0.1 


0.35 


0.54 


0.11 


0.48 


0.48 


0.04 
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MC F 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RM+MED- 
LN-RN-MED+ 

MC F 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

LP M AGE<12 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LM-RN-MED+ 

LP M 11<AGE<40 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LM+RN-MED- 
LN-RN+MED+ 
LN-RH+MED- 
LN-RM-MED+ 

LP M 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LM+RM-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 

LP F AGE<12 

LN+RH+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 



0.54 


0.43 


0.03 


0.46 


0.52 


0.04 


0.57 


0.42 


0.01 


0.48 


0.51 


0.01 


0.62 


0.35 


0.03 


0.53 


0.43 


0.03 


0.65 


0.34 


0.01 


0.35 


0.52 


0.12 


0.27 


0.59 


0.14 


0.4 


0.56 


0.05 


0.31 


0.63 


0.06 


0.43 


0.45 


0.12 


0.34 


0.52 


0.14 


0.48 


0.47 


0.05 


0.18 


0.33 


0.49 


0.13 


0.36 


0.62 


0.27 


0.46 


0.28 


0.2 


0.6 


0.3 


0.22 


0.29 


0.48 


0.17 


0.32 


0.52 


0.33 


0.4 


0.27 


0.39 


0.39 


0.22 


6.31 


0.44 


0.26 


6.47 


0.43 


0.1 


0.38 


0.5 


0.12 


0.47 


0.33 


0.2 


0.38 


0.38 


0.24 


0.66 


0.36 


0.09 


0.16 


0.29 


0.66 


0.12 


0.31 


0.68 


0.25 


0.42 


0.33 


0.19 


0.46 


0.36 


0.2 


0.26 


0.54 


0.15 


0.28 


0.68 


0.31 


0.37 


0.32 


0.31 


0.43 


0.26 


0.23 


0.48 


0.29 


0.38 


0.49 


0.12 


0.3 


0.56 


0.14 


0.37 


0.37 


0.26 


0.29 


0.42 


0.29 


0.46 


0.42 


0.12 
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LP F 11<AGE<40 








LN+RN+MED+ 


0.62 


0.39 


0.09 


LN+RN+MED- 


0.43 


0.46 


0.11 


LN+RN-MED+ 


0.57 


0.39 


0.04 


LN+RN-MED- 


0.48 


0.48 


0.06 


LN-RN+MED+ 


0.6 


0.32 


0.08 


LN-RN+MED- 


0.51 


0.39 


0.1 


LN-RN-MED+ 


0.65 


0.32 


0.03 


LP F 39<AGE 








LN+RN+MED+ 


0.29 


0.4 


0.31 


LN+RN+MED- 


0.22 


0.44 


0.34 


LN+RN-MED+ 


0.37 


0.48 


0.16 


LN+RN-MED- 


0.29 


0.54 


0.17 


LN-RN+MED+ 


0.36 


0.36 


0.3 


LN-RM+MED- 


0.27 


0.39 


0.34 


LN-RH-MED+ 


0.45 


0.41 


0.14 


LD M AGE<12 








LN+RN+MED+ 


0.11 


0.31 


0.68 


LN+RM+MED- 


0.08 


0.32 


0.6 


LN+RH-MED+ 


0.17 


0.47 


0.36 


LN+RN-MED- 


0.12 


0.6 


0.38 


LN-RN+MED+ 


0.13 


0.28 


0.58 


LN-RN+MED- 


0.1 


0.3 


0.61 


LN-RM-MED+ 


0.22 


0.42 


0.36 


LO M 11<AGE<40 








LN+RH+MED+ 


0.27 


0.43 


0.3 


LM+RN+MED- 


0.2 


0.47 


0.33 


LN+RN-MED+ 


0.36 


0.61 


0.16 


LN+RN-MED- 


0.27 


0.67 


0.16 


LN-RN+MED+ 


0.34 


0.37 


0.29 


LN-RN+MED- 


0.26 


0.42 


0.32 


LN-RM-MED+ 


0.42 


0.44 


0.14 


LD M 39<AGE 








LH+RM+MED+ 


0.09 


0.27 


0.84 


LH+RM+MED- 


0.07 


0.28 


0.66 


LN+RN-MED+ 


0.16 


0.43 


0.41 


LN+RN-MED- 


0.11 


0.45 


0.44 


LN-RN+MED+ 


0.12 


0.24 


0.64 


LH-RN+MED- 


0.08 


0.26 


0.66 


LN-RM-MED+ 


0.2 


0.38 


0.41 


LD F AGE<12 








LN+RH+MED+ 


0.2 


0.45 


0.35 


LN+RH+MED- 


0.15 


0.46 


0.37 


LN+RH-MED+ 


0.27 


0.56 


0.17 


LN+RM-MED- 


0.2 


0.61 


0.19 


LN-RN+MED+ 


0.25 


0.4 


0.34 


LN-RN+MED- 


0.19 


0.44 


0.37 


LN-RN-MED+ 


0.34 


0.49 


0.17 



December 1981 - 101 • Ilodgkins Decision Analysts 



LD F 11<AGE<40 

LN+RN+MED+ 0.4 0.47 0.14 

LNfRN+MED- 

LN+RN-MED+ 

LN+RN-MED- 

LN-RN+MED+ 

LN-RN+MED- 

LN-RN-MED+ 

LD F 39<AGE 

LN+RN+MED+ 
LN+RN+MED- 
LN+RN-MED+ 
LN+RN-MED- 
LN-RN+MED+ 
LN-RN+MED- 
LN-RN-MED+ 



0.31 


0.53 


0.15 


0.45 


0.49 


0.06 


0.36 


0.57 


0.07 


0.47 


0.4 


0.13 


0.38 


0.47 


0.15 


0.53 


0.41 


0.05 


0.19 


0.41 


0.4 


0.14 


0.44 


0.43 


0.26 


0.53 


0.21 


0.19 


0.58 


0.23 


0.23 


0.37 


0.4 


0.17 


0.4 


0.43 


0.32 


0.47 


0.21 
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Appendix II - A General Statistical Test For Evaluating Probabilities 

Note: a more extensive discussion of this test, its application and power may be found in Barnett et al, 1981. 

Suppose for a group of N patients numbered from 1 to N according to some neutral procedure (e.g. 
alphabetically), each is known to be in exactly one of M possible states of health. We assume N>M. A 

Bayesian model is used to estimate Pj(j)» the probability that patient i is in health state j (j = 1 M). Given 

that the true state of health for each patient is ultimately learned, how might this information be used to assess 
the validity of the original probabilistic predictions? 

The following procedure allows us to test the null hypothesis Hq mat the predictions are accurate. 

1. Consider the model's estimates for Pj(j). For patient 1 record Pj(l), and for patient 2, P 2 (2), ...for patient 
M, P M (M). For patient M+l, we go back to the first stage and record P^n-l^)' for Patient M+2, 

P M+2 (2), for patient 2M, P2M< M )' for Patient 2M+1, P2M+1* 1 )' ete - Under *** rule ' we choose one of 
the predictions made for each patient, and over the entire set, choose roughly an equal number of predictions 
for each of the M states of health. 

2.Take the N predictions just chosen and list them in decreasing order. Let us define the largest as r^, the 
second largest as r 2 , the smallest as r ^, etc. Note that r^ is not necessarily associated with patient number 1. 

3. Now the patients are to be divided into groups within which the chosen predictions are close together, even 
though, for different people, the predictions may refer to different states of health. For example, if the state 
of health is a stage of Hodgkins disease, one patient may have a 0.4 chance of being in stage I+II, while 
another has a 0.4 chance of being in state III. If these predictions are chosen for study in step 1, both patients 
will be grouped together. 

Begin forming a first group with the patient with prediction r^, the patient with prediction r 2 , etc. Stop, upon 
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reaching K, the smallest number for which 2 n and 2 (1-n) are both at least five. Ifr K . i = r K , 

i=l * i=l r ^ TA ^ 

add the patient with prediction r K+ j to the group. Do the same wfth fjt+j. etc- T*" 8 wffl ensure mat 
people with identical predictions fall in the same groups. 

4. Starting with the next lowest prediction, construct a second group in an analogous way. Proceed similarly 

until as many groups as possible are formed Probably a few patients with die very lowest predictions (e^g. 

-.!:?■- •- . vL. f - -j r jp^ ,i.» ■> ,i / 1 3?'. j t &*fc ^liessifc^i **i u*w £&• Jt*» oi tea. .'$</ "si •'* <s^js£* 
rjj) win constitute a "remainder" not large enough to form a fufl group; they should be added to the last 

- '•>!• • :>•• '.,",-" '. , i- 'TStfJ v:.y;, :",!, .;J*. utiq ; ?:»,.* x.'i .'»' ,Vi V. " .> .-., .• • ' ' 

group formed. 

5. For each patient, note whether OObe was finaHy found to be m me sfi^ of healm wriose estimated 
probability has been fisted. For example, was patient 1 actuary in stage I + II? 

6. Now for each of the various groups formed in steps 3 and 4, calculate the quantity W, defined by: 

, — . x 

where Q * y^^'i for thi'^rbtfp :v: - — < ;i:r 5 ; ; •' — 

A » J^r i (1 - Pl ) 

X ■ the number of group members who were in the state of health 
for which a prediction was made. 
For example, suppose we wish to calculate W for a group of 12 patients wim 



Hodgkins disease for whom one ''has 1 ml* following fiiifQrmaona^ 



Patient Stage for which Recorded Actual 
predictfdW was /""'■ pftJlblftrty ' ■**»<•$•■' 

recorded estimate 

1 I+II 0.25 III 

3 IV 0.25 IV» 

4 I+II 0.26 IV 

7 I+II 0.25 I+II* 

11 I" . R.2« , x+n 

is i+ii 0:25 I+II* 
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0.26 


III 


0.25 


IV 


0.25 


III 


0.25 


I+II 


0.25 


.III* 


0.26 


I+II* 



21 IV 

23 III 

28 I+II 

33 IV 

41 III 

43 I+II 

Note that, since we are considering three states of health for Hodgkins disease (stage I and II, stage III, and 
stage IV), the use of rule 1 leads to the recorded stages shown for the patients listed. 

Here X = 5, because only die cases with a * meet the stated criterion. In this group all the r 's are 025. 

then Q = 12 X 0.25 '3 

and A = (12 X 0.25). 0.76 = 2.25 = 1.6 

Thus W = -2/1.5 = -1.33 for the group. 

Under the null hypothesis of correct predictions, the calculated W should have approximately a unit normal 
probability distribution. 

B 

7. Calculate Z from the rule Z = 2 W:, where W; is the W-value for the jth group, and B the number of 

i=l J J 

groups formed. If the predictions are accurate then Z would be Chi-distributed with B degrees of freedom. 
One can use this fact to determine the significance level (p- value) of the value of Z calculated from the data. 



December 1981 -105- Hotigkins Decision Analysis 



Appendix Hi n Information on Lympbangiogram Used n the Hodgkins Program 



m.l Where Results are Reported as Positue/Equftocal/Negatfre 



SOURCE 



NODES HISTOLOGICALLY POSITIVE 
LAG +vo LAG -va LAG ? 



NODES HISTOLOGICALLY NEGATIVE 
LAG +vo LAG -ye LAG ? 



Aisenberg( '71) 


8 


2 





6 


6 


6 


Enrigtit('70) 


14 


7 


8 


6 


17 


14 


Glees(*7B) 


18 


2 





ft, 


20 


2 


Hanks('72) 


6 








1 


3 





Hass('71) 


16 








1 


27 





Jelliffe('70) 


6 


1 


2 


4 


10 


2 


Kaplan(73) 


61 


6 


6 


id 


137 


43 


Lowenbraun( '70) 


4 








3 


2 





Mitchell ('72) 


10 


2 





11 


16 


2 


Paglia ('73) 


10 


4 





2 


17 


4 


Prosnitz('72) 


13 


4 


2 





10 


7 


Sutcliffe('76) 


22 


4 


2 


2 


64 


6 



187 



31 



19 



67 



321 



86 



False negative ■ 31/237 * 13. IX 
False positive = 67/463 - 12. 3X 



HU Where R«s«lts are Reported as Poskht^NcgBtkeOniy 



SOURCE NODES 


HISTOLOGICALLY POSITIVE 


NODES HIST 


OLOGI 


LAG 


+ve 


LAG - 


■ve 


LAG +ve 


LAG 


Banfi('74) 29 









3 


70 


Castellino('74) 41 









14 


111 


Cotraan('77) 12 




7 




10 


69 


Garcia('71) 2 




4 




2 


7 


Hermreck(1975) 7 




6 




14 


8 


Hellman('74) 26 




2 




12 


38 


Kademian('77) 26 




3 




4 


98 


Wart1re('74) 13 









9 


38 


Urlaub('79) 19 




9 




13 


60 


174 




40 




81 


489 


False negative = 40/214 


= 18. 7% 








False positive ■ 81/670 


« 14. 2X 









-ve 



Overall adding results from both series: 

False negative * 71/451 = 16. 7% 
False positive = 138/1033 = 13.4% 
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Appendix IV - Annotated Trace of Decision Tree Analysis Program in Operation 



In this example, Expand-Top-Level -Choice-Node is called with: 
tests-left = *(IA£) 

prior = '(.SI .17 .02) 

lap-done? ■ NIL 

symptomatology » 'A 

That is, we are considering an A patient with probability 0.81 of being in stage I +11, .17 of being in stage HI, 
and .02 of being in stage IV. Tests other than lyraphangiogram (LAG) and laparotomy (LAP) have either 
been done or are contradicted for other reasons. 

The program trace feature of Lisp has been turned on to show the pattern of function calls which carries out 
the decision tree analysis. The functions Expand-Top-L«vel-Choice-Node, Expand-Chance-Node, 
Expand-Choice-Node, Treatment-vs-LAP, Choose-Best-Treatment, and Treatment-E-U have been selected 
for tracing. To save space and complexity, some caBs to Choose-Best-Treatment and Treatment-E-U at lower 
levels have been deleted. 

(1 ENTER Expand-Top-Level -Choice-Node ((LAG) (0.81 0.17 0.02))) 

First the program calculates, for each treatment, the expected utility <qf carrying out that treatment 
immediately, without further testing. 

1 ENTER Treatment-E-U (TNI (0.81 0.17 0.02))) 

1 EXIT Treatment-E-U 0.77) 

> The utility of immediate total nodal irradiation is 0.77 

1 ENTER Treatment-E-U (MOPP (0.81 0.17 0.02))) 
1 EXIT Treatment-E-U 0.88) 
=> The utility of immediate MOPP chemotherapy is 0.68 

1 ENTER Treatment-E-U (EH (0.81 0.17 0.02))) 
[1 EXIT Treatment-E-U 0.73) 
= => The utility of immediate extended mantle radiotherapy is 0.73 

Next the program calculates the utility of performing LAP followed by optimal therapy defined by 
the results of LAP. 

(1 ENTER Lap-Plan ((0.81 0.17 0.02))) 

(1 ENTER Expand-Chance-Node (LAP (0.81 0.17 0.02) NIL)) 
(1 ENTER Choose-Best-Treatment ((1.0^0.0 0.0))) 
(1 EXIT Choose-Best-Treatment ((0,82 .EM))) 

December 1981 -107- I lodgkins Decision Analysis 



(1 EUTtR Choose-Bis treatment tia.ti 1.0 016)7) 
(1 EXIT Choose-Best-Treatment ((0.64 . TNI))) 
(1 ENTER Choose-Best-Treatment ((0.0 0.0 1.0))) 
(1 EXIT Choose-Best-Treatment ((0.55 . MOPP))) 
(1 EXIT Expand-Chance-NeJ* , 

((0.78 . LAP) utility-data-placerholder 
((I+II 0.81 (1.0 0.0 0>ifi) {{'O'.K .EM))) 
(III 0.17 (0.0 1.0 0.0) ((0.64 .uTIII)» 
(IV 0.02 (0.0 0.0 1.0) ((0.56 . MOPP)))))) 
(1 EXIT Lap-Plan 

((0.78 . LAP) utility-data-place-holder 
((I+II 0.81 (1.0 0.0 0.0) ((0.82 ". EM))) 
(111,0.17 (0.0 1.0 O.0) CCP-64 -t TMJ)» 
(IV 0. tit (0.0 TL6 1.6) ((6.56 v . MOPP)))))) 
= => The utility of LAP is a 78 

Finally, the program calculates the utility of performing lymphangiogram. 

(1 ENTER Expand-Chaece-Nede (LAG (O.BJ. 0*17 O.MMHIl)), 

For the result of LAG being +LAG Ojf^phtmgip^m positw for abdominal nodes), the program 
calculates the optimal subsequent plan and its utility. 

(1 ENTER tx^ahd-Cho+ce-NOde {(0.67 ^.Sft^O.OO) NILf) ! 

(1 ENTER Treatment-vs-LAP ((0.67 0.38 0.05))) 
(1 ENTE* Lap*-*ta* t(0v*7 0.3S *.**)$$< v 

(2 ENTER Expand-Chance-Node (LAP (0.57 0.38 0.06) NIL)) 
(2 EXIT Expand-Chance-Node 

((0.73 . LAP) utility-data-place-holder 
((I+II 0.57 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.39 <0 ? 9 1.0 0,9^((P,64 .,JH})], , ( 
(IV 0.06 (0.0 0"fl iV0)'((OY5§ .MOPP)))))) 
(1 EXIT Lap-Plan 

((0.73 . LAP 5 ) tttHity^data-plaee-ttolder' 
((I+II 0.57 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.38 (0.0 1,0,0,0) ((0,84 ..TNji)) 
(IV 0.05 (0.0 0.8 1.0) ((0.55 . MdPP?))})j 
(1 ENTER Choose-Best-Tr*atment ((0.57 0.38 0:05l)) 
(1 EXIT Choose-Best-Treatment ifOr)! . Htf)lj 
(1 EXIT Treatment-vs-LAP 

((0.73 . LAP) utility-data-place-holder 
((I+II 0.57 (1.0 0.0 0,0) ((0.82 . EM)jj ,, 
(III 0.38 (0.0 1.0 O.'flj f(8?i*' . tfi))f ' 
(IV 0.05 (0.0 0.0 1.0) ((0.55 , MOPP))^))u ,_, 
(1 EXIT Expand-Choice-Node ' 

((0.73 . LAP) utillty-data-Dlace-holder ! 

((i+ii o.57 (i.o d 6 o.of((o;8i . ftf))) 

(III 0,38 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.05 (0.0 0.0 10) ((0.65 .MOPP)))))) 

For the result of LAG being ?LAG (equivocal lymphangiogram), the program calculates the optimal 
subsequent plan and Us utility. 

(1 ENTER Expand-Choice-Node ((0.8? 0.11 O.0T) NIL)) 
(1 ENTER Treatment-vs-LA^ ((0.87 0.12f f.-^" v 
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(1 ENTER Lap-Plan ((0.87 0.12 0>O1))) 

(2 ENTER Expand-Chance-Node (LAP (0.87 0.12 0.01) NIL)) 
(2 EXIT Expand-Chance-Node 

((0.79 . LAP) utility?da*a-pUc#'iftel*er 
((I+II 0.87 (1.0 0.0 0.0) ((0.82 -i i*))) 
(III 0.12 (0.0 1.0 0,0) ((0.84. , TNI))) 
(IV 0.01 (0*0 0.0 1.0) (0,5S -.i MOPP)))):)) 
(1 EXIT Lap-Plan ((0,79 .LAP) utility-dat#Tpl*ce-holder 
((I+II 0.87 (1.0 040 0.0) ((0,82 . EM))) 
(III 0.12 (0.0 1.0 0.0) ((0.64 ta TM))) 
(IV 0.01 (0.0 0.0 1.0) ({0.56 .MOPP )))))) 
(1 ENTER Choose-Best-Treatment {(0.|7 0,12 0. 01))) 
(1 EXIT Choose-Be$t-Treajtment ((0,18 . TNI))) 
(1 EXIT Treat«ent-v*-LAP 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.87 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.12 (0.0 1.0 0.0) ((0.64 .TNI))) 
(IV 0.01 (0.0 0.0 1.0) ({0.66 i MOPP)))))) 
(1 EXIT Expand-Choice-Node ((0.79 . LAP) utility-data-place-holder 
((I+II 0.87 (1.0 0.0 0.0) ((0.82 . E*H) 
(III 0.12 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 

For the result of LAG being -LAG (negative lymphangiogram), the program calculates the optimal 
subsequent plan and its utility. 

(1 ENTER Expand-Choice-Node ((0.89 0.10 0.01) NIL)) 
(1 ENTER Treatment-vs-LAP ((0.89 0.10 0.01))) 
(1 ENTER Lap-Plan ((0.89 0.10 0.01))) 

(2 ENTER Expand-Chance-Node (LAP (0.89 0.10 0.01) NIL)) 
(2 EXIT Expand-Chance-Node 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 
(1 EXIT Lap-Plan 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 .EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.56 . MOPP)))))) 
(1 ENTER Choose-Best-Treatment ((0.89 0.10 0.01))) 
(1 EXIT Choose-Best-Treatment ((0.79 . TNI))) 
(1 EXIT Treatment-vs-LAP 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 
(1 EXIT Expand-Choice-Node 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.84 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 

The lymphangiogram plan is complete. Regardless of the result of LAG, as shown in the plan 
fragment below, the best action to lake subsequently is laparotomy. Hence, LAG can be omitted, 
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as the plan beginning with with LAP will be better in all respects. 

(1 EXIT Expand-Chance-Node 

((0.77 . LAG) utility-data-place-holder 
((+LA6 0.25 (0.57 0^8 8.86) 

((0.73 . LAP) utility-data-plac«-hold«r 
((I+II 0.67 (1.6 0.90.9) ((0.82 ; t*))) 
(III 0.38 (0.8 1.0 0.0) ({6.84 >i T»I))) 
(IV 0.05 (0.0 8.0 1.8) ((0.55 . MOPP)))))) 
(?LAG 0.22 (0.87 0.12 0.01) 

((0.79 . LAP) utillty-data-place-holder 
((I+II 0.87 (1.0 0.8 0.0) ((8^82 '• . E»))) 
(III 0.12 (0.0 1.0 0.8) ((0.84 .Tim))} 
(IV 0.01 (0.0 0.0 1.0) ((0.65 . *»PP))))t) 
(-LAG 0.53 (0.89 0.10 8.81) 

((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 8.0) ((0.8* J EH})) 
(III 0.10 (0.0 1.0 0.0) ((8.64 . TM))) 
(IV 0.81 (0.0 0.8 1.0) ((8.66 . HOPP))))))))) 
(1 EXIT Expand-Top-Level-Choice-Hode 32} 
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